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Dear School Science and Mathematics Journal Readers 





In my effort to help “Fight the Fight for Public School 
Teachers,” I have written annual editorials about teachers or 
teacher educators: Cindy Perdue, who taught kindergarten 
for 38 years in New Castle, Indiana; Maureen Reichardt, 
who after 33 years as a public school teacher resigned her 
position and is now at Guerin Catholic High School in 
Noblesville, Indiana; and Dr. Beatriz D’ Ambrosio, a former 
mathematics teacher educator at Miami University in 
Oxford, Ohio, who worked to honor and support public 
school teachers. This will be the fourth such editorial. 

Please let me introduce you to Halla Shteiwi. Halla 
teaches mathematics and is the Mathematics Department 
Head at Hughes STEM High School (HSHS), located 
diagonally across the street from my office at the Univer- 
sity of Cincinnati (UC). [Full disclosure: Ms. Shteiwi was 
a student in courses I taught at UC.] I interviewed Halla at 
HSHS in her classroom after school and audio-recorded 
the conversation (November 13, 2017). 

When asked, “Why did you want to be a teacher?” Halla 
explained. Her father wanted her to work in his business 
so, therefore, business was her first major but she “wasn’t 
learning anything.” Her next major was law, which 
required “too much reading.” Not at all sure about where 
to turn next, she began studying in the Math Lab at UC 
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and at times, helping other students with their mathemat- 
ics homework. The person in charge of the Math Lab 
asked her if she wanted to work there and his question 
made her think: “Maybe I should be a math teacher.” 
Before a final decision on her major, she took one educa- 
tion course and was assigned a field experience. While in 
the field experience classroom, she was supposed to just 
observe but that seemed impossible when so many stu- 
dents needed help. She broke with protocol and helped the 
student sitting at the desk next to hers and realized: “This 
is for me!” 

That was the beginning of her quest to become a teacher. 
Ms. Shteiwi earned dual UC degrees in December 2009, a 
B.S. in Secondary Education, and a B.S. in Mathematics. 
During the summer of 2010, she took a trip to her parents’ 
birthplace, Jordan, and did not return before the start of the 
2010-2011 academic year. In October of 2011, I heard that 
an HSHS teacher had quit and suggested Ms. Shteiwi as a 
possible substitute. Ultimately, she was hired and began to 
teach in January of 2011. Halla described her first semester 
at HSHS as difficult and stressful. Accordingly, the students 
had “no manners.” This was not how she was raised. She 
recalled screaming at them and even telling them to shut up. 
Yet, she ultimately learned to change her tone of voice, ask 
questions rather than offer advice, and “never yell at a kid.” 
Giving credit to her colleagues, Mr. Lane, Ms. Vagedes, 
and Mr. Frecker, for these lessons, Halla said she ‘‘continued 
to grow.” Running an effective classroom is not an issue for 
Halla now, but she was quick to say, “They [the students] 
do not like me at the beginning of the year and they learn to 
love me at the end.” 

With classroom discipline no longer an issue, Halla 
focused on establishing her teacher identity. In fact, one 
aspect of her identity is her belief that a// students can 
learn and her role is to make that happen. I asked her, 
“Why do you have a stance that a// kids can learn?” Hal- 
la’s succinct answer: “My background.” Halla’s parents 
were “extremely poor” when they immigrated to America 
from Jordan. Her father completed the equivalent of a 
sixth-grade education and her mother completed the 
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equivalent of a 10th-grade education. Rakan Shteiwi 
worked three different jobs with the commitment that his 
children were not “going to be like me.” Serendipitously, 
while working as a waiter Rakan became friendly with a 
banker, who loaned him money to start a restaurant. Using 
his first business profits from this restaurant, Mr. Shteiwi 
paid the loan back and opened more restaurants. Halla 
said her father instilled these things in her: “If you are sick 
you are still going to go to school unless you are throwing 
up or bleeding.... Do everything you need to be on the 
honor roll.... No after-school activities unless you main- 
tain your grades.” She recalled a time when she asked for 
a calculator and her father said, “Absolutely not.” Halla 
and her siblings were required to learn English and 
Arabic. Although her father worked every day of his life, 
he ritually came home in the evenings during homework 
time to help his children with Arabic and school home- 
work. When Halla was 15, she started working in the 
family-owned restaurant and continued to work there, 
either full-time or part-time, for the next 10 years. Work 
ethic and a “no excuse” philosophy instilled by her father 
are reasons Halla believes a// students can learn. Failure is 
not an option. 

Let me interrupt Halla’s story with my observations. I 
would describe Ms. Shteiwi as a warm demander (Bondy, 
Ross, Hambacher, & Acosta, 2008). The description 
warm demander first appeared in Kleinfeld’s (1975) study 
when he used it to portray the ways in which effective 
teachers of Eskimo and Alaskan Indian children interacted 
with their students. Warm demanders “embrace values 
and enact practices that are central to their students’ 
success” (Bondy et al., 2008, p. 420). Bondy and Ross 
(2012) encapsulated a warm demander teacher as one who 
treats “students with unconditional positive regard, know- 
ing students and their cultures well, and insisting the stu- 
dents perform to a high standard” (p. 58). Ware (2006) 
used two terms to describe a warm demander: caregiver 
and authority figure. 

Like Halla’s father, warm demanders establish a “no 
excuses” approach (Wilson & Corbett, 2001). They will 
not accept students’ excuses for failure, but more criti- 
cally, they blame themselves if their students fail. Bondy 
and Ross (2012) described practices that warm demanders 
enact: “Build Relationships Deliberately; Learn About 
Students’ Cultures; Communicate an Expectation of Suc- 
cess; [Move] Beyond Believing to Insisting; Provide 
Learning Supports; Support Positive Behavior; and, Be 
Clear and Consistent with Expectations” (pp. 2-6). (Please 
read the entire article for more details of each of these 
practices.) 


a 


Let me return to Halla’s story by retelling one of her 
stories of a former student, Tamra (pseudonym), who is 
now about 21 years old. When Tamra was a student in 
Halla’s mathematics classroom, Halla found out that 
Tamra’s home life was in shambles. Tamra was taking 
care of her siblings because her mother moved to another 
city in another state. Halla told Tamra, “If you need time 
away you can always come with me.” Halla got parent 
consent and took Tamra to Halla’s parents’ home for a 
three-day weekend. They played family games, watched 
movies, ate meals together, and relaxed. On the way back 
to school Monday morning, Tamra told Halla: “If my 
whole life could be like the weekend I just had I would 
have no troubles in the world. I could succeed at every- 
thing I did.” She was accepted into college with scholar- 
ships but chose not to attend because of pressure from her 
mother. Despite this, Halla and her husband, Nathan, have 
continued to mentor Tamra through the ups and downs of 
life. They recently helped Tamra furnish an apartment 
with a “couch, vacuum cleaner, dining room table and 
chairs, dishes” and other items. Tamra originally wanted 
to pursue a career in law enforcement but is now working 
for an in-home health care company. Ms. Shteiwi and her 
husband built a strong relationship with Tamra that has 
lasted many years. Recall that warm demanders 
“deliberately build relationships” with students (Bondy & 
Ross, 2012). 

The perception that relationships end when students 
leave our classrooms is a myth for Halla (and many public 
school teachers!). Teaching days are short and teachers 
have summers off are also myths for teachers like Halla. 
On a typical workday “since baby” (darling Laila is not 
quite two years old), Halla rises at 5:30 a.m. and arrives at 
HSHS between 7:00 and 7:30 a.m., this while sometimes 
“acting out the lesson ... thinking of questions to ask stu- 
dents ... while driving.” Before students arrive, she sets 
up technology or other equipment, assigns students to 
groups, and arranges the desks for the group work. During 
her planning bell she writes lesson plans with her student 
teacher. As added duty because of her role as Mathematics 
Department Head, Halla observes and evaluates mathe- 
matics teachers in her department. She also does so during 
her planning bell. 

School ends at 3:25 p.m. but the work does not stop 
then. On Monday, Tuesday, and Wednesday she stays late 
to grade papers, review exit tickets, schedule tutors, and 
make Show Me™ apps for days she needs a substitute 
teacher, just to name a few tasks. When asked about how 
many students she works with each day she told me, “134 
students on Monday, Tuesday, and Friday. 96 students on 
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Wednesday and about 43 students on Thursdays.” On 
Thursdays students do iWorks'™ which is an online tuto- 
rial system; they work on individualized mathematics 
skills that Halla assigns each week. All of this while Halla 
is “constantly thinking, How can I make it better?” 

So, back to me. If I feel isolated in my so-called “ivory 
tower” all I need to do is walk across the street and Halla 
welcomes me into her classroom. One time I when I was 
visiting another teacher she saw me in the hallway and 
asked me to come into her class. She introduced me to her 
students: “This is Dr. Harkness. She was my teacher.” 
Well, if you have ever been a teacher you can appreciate 
how I felt when I was introduced as a teacher, not a pro- 
fessor mind you, a teacher. 

This story has provided a glimpse of the life of one pub- 
lic school teacher, Ms. Shteiwi. 

Her story is one of warm demanding. Care. Authority. 
Not accepting excuses. Having high expectations. Build- 
ing relationships. A stance that a// students can learn. She 
learned some of these lessons from her father, Rakan 
Shteiwi, an immigrant from Jordan. She learned some les- 
sons from dedicated colleagues. And in her own words, 
she is “still growing” as a mathematics teacher in an urban 
public school. 

To end, let me quote Diane Ravitch (2013), former 
assistant U.S. Secretary of Education: 


Public policy should leverage federal and state funding 


to reduce poverty and racial isolation in schools 
...preserving public education and strengthening it 
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rather than privatizing it is critically important to 
preserve our democracy. And blaming and punishing 
the people who work in our neediest public schools for 
the persistence of social, economic, and educational 
inequities is not just wrong, but will further damage the 
children in whose interest the corporate reformers claim 
to be acting. We do not help children by demoralizing 
those who do the most difficult work with them under the 
most difficult conditions, every day. (p. 299; italics 
added for emphasis) 


Children need teachers like Ms. Shteiwi. I was her 
teacher and she has been mine. 

Warm Regards, 

Shelly Sheats Harkness 

University of Cincinnati 
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In the transition to middle school, and during the middle school years, students’ motivation for mathematics tends to 
decline from what it was during elementary school. Formative assessment strategies in mathematics can help support 
motivation by building confidence for challenging tasks. In this study, the authors developed and piloted a professional 
development program, Learning to Use Formative Assessment in Mathematics with the Assessment Work Sample Method 
(AWSM) to build middle school math teachers’ understanding of the characteristics of high-quality formative assessment 
processes and increases their ability to use them in their classrooms. AWSM proved to be feasible to implement in the 
middle school setting. It improved teachers’ practice of formative assessment, especially in their feedback practices, 
regardless of their pedagogical content knowledge at entry. Results from focus groups suggested that teachers were better 
able to implement ungraded practice and student self- and peer-assessment after AWSM, and that students were more 


willing to engage in complex problem solving. 


In the transition to middle school, and during the middle 
school years, students’ motivation for mathematics tends to 
decline from what it was during elementary school. At this 
age, students report less valuing of mathematics and lower 
effort and persistence in mathematics problem solving over 
time (Pajares & Graham, 1999; Valas, 2001; Wigfield, 
Eccles, & Pintrich, 1996). Middle school students also 
report lower confidence in their mathematics ability than 
before (Clarke, Roche, Cheeseman, & van der Schans, 
2014; Pintrich & Schunk, 1996), influenced in part by 
exposure to a larger peer group with whom they begin to 
compare themselves, more perceived competition in the 
classroom environment, and more rigorous standards for 
evaluation (Eccles & Midgley, 1989). Some students 
perceive that they are judged more on innate ability in 
middle school than on improvement (Anderman & 
Midgley, 1997), and come to believe that if they lack 
ability, their effort will not help (Valas, 2001). Students 
also lose confidence that they can self-regulate, in that they 
feel less able to organize their thoughts, understand 
complex tasks, and choose and evaluate problem-solving 
strategies (Pajares & Graham, 1999). 

Low Confidence in Mathematics Affects Teaching and 
Learning 

These feelings have an effect on teaching and learning in 
the middle school mathematics classroom. Students who 
are not confident that they can solve complex problems, or 
who do not see the point of putting forth effort to do so, try 
to avoid those tasks or pressure teachers to make the work 
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simpler for them (Clarke et al. 2014). This lack of 
mathematics self-efficacy is a predictor of lower 
mathematics achievement outcomes (Pajares & Graham, 
1999). Some middle schoolers will attempt to engage in 
mathematics learning only when tangible rewards are 
offered (Rowan-Kenyon, Swan, & Creager, 2012). 
Students’ difficulties with motivation and persistence, 
therefore, become a problem of practice for middle school 
teachers. 

In this environment, middle school mathematics teachers 
can feel hesitant giving students challenging and complex 
work. If they do so anyway and students encounter 
difficulty, some teachers oversimplify the task or tell 
students how to solve it (Clarke et al., 2014; Ferguson, 
2009). This is especially true when teachers work with 
lower-achieving students (Zohar, Degani, & Vaaknin, 
2001). However, the Common Core and_ other 
contemporary U.S. mathematics standards require that 
students be able to solve complex problems and to explain 
their reasoning, so teachers need effective instructional 
strategies to support students in these practices. 

Instruction Can Build Confidence for Challenging 
Mathematics Tasks 

Research has supported some features of instruction and 
the classroom environment that make it easier for middle 
school mathematics teachers to pose challenging tasks and 
for students to engage, persist, and succeed in them. Rather 
than lower the level of cognitive demand of difficult 
problems, teachers have found success by supporting 
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Task Selection 





Clear learning goals and an aligned 
student task are identified. 





Students take action as 
part of the continuous 
learning process. 
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Figure 1. AWSM formative assessment process. 








small-group discussions and peer collaboration (Ferguson, 
2009; Rowan-Kenyon et al., 2012; Ryan & Patrick, 2001; 
Schunk & Pajares, 2001; Zohar et al., 2001). When 
teachers demonstrated to students that they are supportive 
and will try to help students as individuals, and when 
teachers had clear expectations that students would respect 
one another’s ideas in the classroom, students were less 
disruptive and better able to self-regulate (Ryan & Patrick, 
2001; Valas, 2001). A classroom focused more on mastery 
and improvement rather than competition also supported 
students’ academic confidence (Ryan & Patrick, 2001). 
Formative Assessment Strategies Can Help Students 
Be More Confident Mathematics Learners 

Formative assessment is an evidence-based process of 
gathering information on three questions: (a) Where am I 
going? (b) How am I doing now? (c) Where do I go next? 
to support a learning cycle (Hattie & Timperley, 2007; 
Sadler, 1989). Therefore, the most important formative 
assessment practices involve (a) students’ understanding of 
their learning goals and targets, (b) the criteria by which 
they will know how they are progressing with their 
learning, and (c) what needs to be done next to move 
learning forward. Feedback is an active part of the process 
and can address the task, the student’s processing of the 
task, suggestions for what to work on next, and scaffolds 
for the individual student (Hattie & Timperley, 2007). 
Literature supports prioritizing these three dimensions of 
formative assessment (see Figure 1). 

Assessment-centered teaching can have profound effects 
on student learning and motivation (DiRanna et al., 2008). 
Well-designed formative assessment is associated with 
major gains in student achievement across all ages and 
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subjects, and has its greatest positive impact on students 
who struggle in mathematics (Black & Wiliam, 1998a, 
1998b). To understand learning targets, students need to 
have clear knowledge of where they are going with their 
learning and avoid the inefficiency and frustration of trial 
and error (Sadler, 1989). Students who struggle in school 
particularly benefit from understanding learning targets, 
because they do not intuit the targets on their own (Black & 
Wiliam, 1998a, 1998b; James et al., 2006). Clear success 
criteria enable students to understand how they are doing 
on learning targets, so they can judge and self-regulate the 
quality of their work (e.g., Andrade, Du, & Mycek, 2010; 
Brookhart, Andolina, Zusa, & Furman, 2004; Sadler, 
1989). When teachers involve students in the assessment 
process, students perceive more control of and more 
responsibility for their own learning (Rieg, 2007). Allowing 
students to help generate success criteria gives them a 
feeling of empowerment and makes assessment of their 
work seem less punitive and more constructive (Brookhart, 
1997). In turn, students feel more competent and are more 
likely to engage in learning (Stiggins, Arter, Chappuis, & 
Chappuis, 2006). 
Formative Assessment Is Particularly Difficult in 
Mathematics Without Support 

Formative assessment requires teachers to elicit students’ 
existing ideas as students make their thinking visible to help 
further understanding. In mathematics, it is often difficult to 
interpret students’ understanding, knowledge, and learning 
of complex content. Interpretations of student work as 
evidence of learning may be powerfully influenced by 
teachers’ own understandings of mathematics concepts and 
how the concepts can be communicated, and expectations 
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regarding the performance of whole classrooms and 
individual students (Even, 2005; Morgan & Watson, 2002). 
Professional Development in Formative Assessment in 
Mathematics Needs Improvement 

Much of current teacher professional development (PD) 
in formative assessment is delivered in the form of one-time 
workshops, which are ineffective at changing teachers’ day- 
to-day professional practices (McLaughlin & Talbert, 
2006). Even more extensive PD that includes books and 
videos does not necessarily help teachers change their 
practice. In a study of one such program, implementation 
was low, and while teacher knowledge grew, teacher 
practice did not change (Randel, Beesley, Apthorp, Clark, 
& Wang, 2016; Randel et al., 2011). Assessment work 
sample method (AWSM) is intended to support 
mathematics teachers in using, not just learning about, 
high-quality formative assessment strategies linked to the 
overall formative assessment process. 


Program 

Learning to Use Formative Assessment in Mathematics 
with the AWSM is a PD program that builds middle school 
mathematics teachers’ understanding of the characteristics 
of high-quality formative assessment processes and 
increases their ability to use them in their classrooms. We 
were inspired to create AWSM following the results from 
previous research (Randel et al., 2011, 2016) on a formative 
assessment PD program developed for all content areas and 
levels of learning. This program did not change teacher 
practice in mathematics, in part because it had few 
examples of how to implement formative assessment in that 
content area. By contrast, AWSM provides PD that builds 
formative assessment practices and skills specifically in 
mathematics. AWSM is about helping teachers develop 
effective instructional practices embedded in everyday 
teacher lessons and uses actual student and teacher work 
along with peer review/support. 

In AWSM, reviewing and discussing samples of student 
work help teachers shift from thinking of teaching as 
something teachers do to a focus on learning as something 
students do, because it is in student work that student 
thinking is made visible (e.g., Hattie, 2009). The AWSM 
approach uses the Assessment Work Sample Method, a way 
to collect, create, discuss, and learn about a formative 
assessment process using student work samples that help 
ground the learning in daily practice (Clare, Valdés, Pascal, 
& Steinberg, 2001; Matsumura et al., 2006). The work 
samples include a teacher cover sheet that conveys the 
goals of the lesson, the type of knowledge/skill being 
developed, the success criteria for meeting learning goals, 
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and general information that will help reviewers understand 
the “what and why” of the assignment. Attached to the 
cover sheet are four pieces of student work, two that 
achieved the learning goals and/or targets and two that did 
not (see Figure 2 for work sample excerpt). In AWSM, 
work samples in each session are the foundation of 
discussions around formative assessment and mathematics 
teaching and learning. 

AWSM Includes Recommended Characteristics of PD 
in Formative Assessment 

High-quality PD in formative assessment should be 
intensive and ongoing, connected to practice, collaborative, 
content-focused, adapted to local context, active, 
systemically supported, and coherent (Trumbull & Gerzon, 
2013). It should offer teachers choices about the 
instructional practices they will focus on, the flexibility to 
adapt formative assessment to their context, provide 
teachers with a lot of time to change practices, and offer 
both support and accountability from peers (Leahy & 
Wiliam, 2012). AWSM has these recommended features, as 
well as a content focus on mathematics. Specifically, 
AWSM places teachers in collaborative learning groups, 
features ongoing meetings throughout the year, and uses 
work samples to connect the PD to content, instructional 
practice, and local context. The sessions aim to prompt 
teachers to examine their current assumptions and decide 
which aspects of their practice to change, provide time to 
learn new approaches, and stimulate reflection that leads to 
strategy implementation. 

AWSM is structured around nine face-to-face meetings 
which include a two-day introductory workshop and eight 
sessions of about 45 minutes each. The introductory 
workshop is designed to help participants gain a shared 
understanding of the formative assessment process and 
prepare to create positive classroom environments while the 
eight short sessions are designed to support teachers as they 
implement the process. During Part One of the introductory 
workshop, participants build their understanding of 
formative assessment as one component of a larger 
assessment system, with an emphasis on providing 
descriptive feedback to move the learning forward. 
Participants discuss the characteristics of positive classroom 
culture and why it is essential, and consider structures that 
will help them create such a culture. Participants learn 
about fixed and growth-oriented mindsets (Dweck, 2006) 
and the type of feedback that will promote a growth- 
oriented mindset that builds student self-efficacy for 
mathematics. They discuss physical, social, and emotional 
factors that impact classroom culture and share successes 
and challenges of creating such cultures in their classrooms. 
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Teacher Cover Sheet 


5. What were your learning goals for the students for this assignment? In other words, 
what skills , concepts, or facts did you want students to learn, practice or demonstrate 


understanding of as a result of completing this assignment? (Students will know and 
understand that: ...) 


| nSUA an | LAS 
) I | ‘ / ) f 
ANE ANA Vl AVG TO CAiCwWate UN ANG ZAG, Ur Ve AVE 
U 
0 ond OnS g Lhe rates Were no Che Sd 


6. Check the type of pearmsnang on) tenet this assignment addresses (check all that apply): 
Knowledge (facts/details to be memorized) 
MY Skill (algorithmic procedures) 
Conceptual Understanding (reasoning, generalizing, 
explaining, etc.) 
(W Problem Solving within a Context (multiple procedures; 
solution strategy) 


11. a. How was this assignment assessed? If there is a rubric, student reflection, etc., please attach it. If 
you are not attaching a rubric, please explain your criteria for determining if students met the 
learning goal of the assignment. 


The rubric is attached (on back. of Assi nent). 








11. b. Did you share these criteria with the students? -4Yes [] No 
Student Work 
pais 4 
aT wae 7a 
x =.1¢ 
\fod 
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Figure 2. AWSM work sample. 


In Part Two of the introductory workshop, participants 
plan for implementing formative assessment by first 
examining authentic student work. In collaborative groups, 
teachers analyze the intended learning goals and success 
criteria from anonymous work samples and then compare 
the teachers’ intended learning goals to accompanying 
student work. Using criteria, participants determine if the 
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teacher’s intended learning and the actual student tasks are 
strongly aligned, partially aligned, or weakly aligned. 
Mathematical content as well as the inferred cognitive 
demand of both the learning goal and student task are 
reviewed and discussed. Through this analysis, participants 
clarify their understanding of Dimension | (learning goal 
and aligned student task) and Dimension 2 (success criteria 


if 
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aligned to the learning goal and student task). These 
dimensions are the foundation for the formative assessment 
process because without clarity about what is to be learned 
and clear criteria for goal attainment, the feedback process 
can be derailed and learning impeded. When these 
dimensions are clear, students are supported in taking 
greater ownership for learning. 

During Part Three of the introductory workshop, 
participants apply the learning from Parts One and Two to 
their own instructional planning. During this segment, 
participants analyze an instructional unit they plan to 
implement to determine whether the learning goals, student 
tasks, and success criteria are strongly aligned. Participants 
consider strategies for how to best communicate learning 
expectations to students and identify/adapt tools to help 
students track their own progress. Independent work time 
allows participants to initiate revisions to their instructional 
unit if needed and receive feedback from peers and 
workshop facilitators. 

The short sessions are scheduled around grade level team 
times and are organized as teacher learning communities 
with a facilitator that has both mathematics and formative 
assessment expertise. Dimension 3, providing descriptive 
feedback, is the focus of Sessions 1-4, but the program 
continues to reference topics from the introductory 
workshop such as class culture, student attitudes, clarity of 
learning goals, and success criteria because it is important 
for participants to understand how these dimensions 
intersect. Participants learn what effective descriptive 
feedback entails and how to provide both oral and written 
feedback. Cues, questions, and recommendations for next 
steps help students take more responsibility for learning 
than simply providing correct solutions or step-by-step 
actions. Participants also learn that students should be 
active in the feedback process, and strategies for 
implementing effective peer- and self-assessment are 
discussed. In Sessions 1-5, anonymous student work 
samples remain at the core of the PD. Participants assess 
work samples to determine the degree to which written 
teacher comments will move student learning forward. 
During this process participants make clear connections 
between AWSM Dimensions 1-3, as providing effective 
feedback is dependent on clearly articulated learning goals 
and success criteria. Participants examine work samples 
that incorporate student peer and self-assessment, and 
they use the AWSM rubric (adapted from Matsumura, 
Garnier, Pascal, & Valdes, 2002) to independently assess 
anonymous work samples and practice providing written 
feedback. 
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Rather than analyze the student work of an unknown 
teacher, Sessions 6-8 ask participants to share their 
instructional practice more directly. In these sessions, 
participants generate their own student work samples, 
present the work to colleagues, and request feedback on 
how to improve their own implementation of the process. 
These sessions are powerful as the entire formative 
assessment process is fully integrated. Participants reflect 
on their own progress for implementing formative 
assessment and identify next steps. The formative 
assessment process is modeled during the each of the 
sessions. Clear goals and success criteria are provided, 
procedures are modeled and discussed, and feedback is 
incorporated to support learning among the participants. 
Clarifying this approach and calling out the modeling helps 
participants think about what their students need from them 
to engage in an ongoing feedback process. 

In AWSM, the expertise of the session facilitator should 
not be underestimated. The mathematics teachers bring 
targeted questions to these sessions, and the facilitator must 
be prepared to reference research, share how they 
themselves were able overcome challenges with using 
formative assessment, and continue to build and maintain 
trust with participants. 


Methods 

In school years 2012-2013 and 2013-2014, we worked 
with seven middle schools in an urban district in Colorado, 
with a total of 47 mathematics teachers. One pilot school 
received the AWSM PD in the 2012-2013 school year, and 
the remaining six schools received the PD in the 2013— 
2014 school year. 
Research Questions 

The overall evaluation of AWSM_ addresses three 
primary research questions: (a) to what extent can AWSM 
be implemented with fidelity in an authentic education 
delivery setting? (b) to what extent does AWSM show 
promise for improving teacher practice of mathematics 
formative assessment? and (c) to what extent does AWSM 
show promise for increasing student achievement in 
mathematics? The research involved teacher measures of 
formative assessment practice, student measures of 
mathematics achievement, observations of professional 
development sessions, and teacher questionnaires and focus 
groups. 
Data Collection 

To answer the aforementioned research questions, data 
were collected from (a) teacher work samples for assessing 
teacher practice in formative assessment, (b) a pretest of 
content and pedagogical knowledge in mathematics, (c) a 
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Goals are highly 
focused on student 
learning. Goals are: 


fully elaborated (all 
aspects and parts of 
the learning goal are 
included and 
described); 
appropriate to 
student learning 
needs (grade/course 
appropriate); and 
Clearly 
communicated to 
students. 


Assessment criteria are: 


communicated so 
that students clearly 
understand the 
assessment criteria 
(with detailed 
elaborations) for high 
quality, medium 
quality, and low 
quality work; 

made explicit through 
oral dialog, written 
criteria, and/or 
leveled examples of 
student work; and 
discussed with 


Goals are focused on 
student learning. 
Learning goals: 

e are clear statements 


(not an activity, topic, 
phrase, or question); 


© are appropriate to 


student learning 
needs (grade/course 
appropriate); and/or 


e may or may not be 


clearly communicated 
to students. 


Assessment criteria are: 
¢ communicated so 


that students 
understand the 
assessment criteria 
(but without detailed 
elaborations) for 
high quality, medium 
quality, and low 
quality work; and 
communicated 
through at least 
written instructions 
but may also provide 
examples and oral 
instructions. 


Goals are somewhat 
focused on student 
learning. Goals: 

e are stated as 


activities or questions 
with no clear 
description of what 
students should 
know or be able to 
do; 

may not be 
appropriate to 
student learning 
needs (grade/course 
appropriate); or 

may not be clearly 
communicated to 
students. 


Assessment criteria are: 
© communicated so that 


students know what 
they must do, but the 
criteria do not 
delineate between 
what high quality, 
medium quality, or 
low quality work 
involves (e.g., may 
include a checklist or 
rubric based on 
presence or absence 
of work elements). 


No learning goals are 

provided for the task. 

e noclear description 
of what students 
should know or be 
able to do; or 

e topic rather than 
learning goal may 
be listed. 


No assessment criteria 


are stated or it is not 


possible to determine 


the assessment criteria 
from the work sample 


documents. 





students. 





Figure 3. Sample of two dimensions of AWSM work sample rubric (adapted from Matsumura et al., 2002). Higher scores indicate higher formative 


assessment quality. 


district-wide mathematics assessment to assess the impact 
of AWSM on student mathematics achievement, and (d) 
teacher focus groups to assess teacher perceptions of the 
impact of AWSM on their knowledge and practice of 
formative assessment. Data collection for each of these 
outcomes measures are discussed in turn below. 

Work samples. To provide a measure of teacher 
practice of formative assessment, participating teachers 
provided samples of student work that were graded and/or 
marked with teacher feedback. Student work samples were 
linked to an assignment that could be a homework/seatwork 
assignment, performance assessment, or quiz/in-class 
assessment. For each assignment, teachers completed a 
cover sheet to describe the assignment (including directions 
given to students); the source of the assignment (e.g., 
teacher created; from a textbook, workbook, curriculum 
publisher, etc.); why the particular assignment was selected; 
the purpose of the assignment (e.g., check student progress; 
provide practice, unit post-test); the student learning goals 
for the assignment; student accommodations for the 
assignment (e.g., addressing different skill levels; whether 
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students received help); how the assignment was assessed 
(e.g., rubric; grading system; student reflection); and criteria 
for deciding whether students met learning objectives. For 
each assignment, participating teachers provided four 
samples of student work—two that met learning objectives 
and two that did not meet learning objectives. 

Participating teachers submitted work samples at the 
beginning of the school year, prior to the AWSM 
professional development sessions (pretest), and again at 
the conclusion of AWSM program (post-test). Two analysts 
both scored all work samples using the AWSM rubric (see 
Figure 3 for sample) on a scale from 1 to 4, indicating the 
level of quality of formative assessment practices on six 
separate dimensions: (a) focus of the goals on student 
learning; (b) alignment of learning goals and task; (c) 
alignment of learning goals and assessment criteria; (d) 
clarity of the student assessment criteria; (e) feedback type; 
and (f) feedback integrates student involvement. When 
scores for the two raters were discrepant, the raters 
discussed the work sample and reconciled their scores to 
produce one score per work sample per dimension. 
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Measure of pedagogical content knowledge. The 
learning mathematics for teaching (LMT) assessment 
(Learning Mathematics for Teaching Project, 2011) was the 
measure of teacher content and pedagogical knowledge in 
mathematics. The purpose of the LMT assessment is to 
capture teachers’ mathematical knowledge for teaching 
(MKT). The LMT is not an assessment of “common” 
content knowledge alone, but knowledge that is specific 
and useful for teaching student mathematics. The 
assessment was designed to discriminate teachers’ MKT 
across the entire ability range; therefore, items were chosen 
such that the average teacher answers 50% of them 
correctly. The developers stressed that the assessment not 
for drawing conclusions about the competency of 
individual teachers, but rather for purposes such as 
measuring the effectiveness of professional development. 
The LMT content area assessments included Middle School 
Number Concepts and Operations (NCO) and Middle 
School Patterns, Functions and Algebra (PFA), as these 
assessments were aligned to AWSM PD content. 

Student achievement. For the impact analysis of the 
AWSM on student achievement, the outcomes were 
mathematics scores on the Measures of Academic Progress 
(MAP) assessment administered at two-time periods during 
the school year (fall and spring). We used a quasi- 
experimental difference-in-differences approach (Cook, 
Shadish, & Wong, 2008), comparing students of teachers 
who participated in the AWSM PD to their previous year’s 
students prior to AWSM exposure. Two separate analyses 
were conducted for proximal and distal outcomes. The 
proximal outcome analysis was conducted at the end of the 
training year and included 7 schools, 47 teachers, 2,281 
counterfactual group students, and 3,896 treatment group 
students. The distal outcome analysis was conducted one- 
year post-training and included 1 school, 3 teachers, 312 
counterfactual group students, and 330 treatment group 
students.’ Both the proximal and distal analyses utilized 2- 
level hierarchical linear models (HLM) to account for 
students nested within teachers. The proximal analysis 
included school dummy codes at level 2 to account for 
shared variance between students at the school level. 

Teacher focus groups. Schools were to have two 
teacher focus groups in the fall 2013 semester and two in 
the 2014 spring semester. All but two schools had the four 
planned focus groups; due to scheduling difficulties two of 
the schools had three focus groups, resulting in a total of 22 
focus groups for the 2013-2014 school year. Each focus 
group lasted for about 45 minutes. The focus groups were 
audio recorded and transcribed. 
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The semistructured focus group protocol addressed the 
research questions by asking for teachers’ feedback about 
AWSM_ implementation (“How have the program 
facilitators supported you before and during AWSM?”), 
perceptions of impact on their classroom practice (“To what 
extent is AWSM building your ability to use formative 
assessment in your classroom?”’), perceptions of impact on 
student learning (“What strategies seem most effective for 
student learning?”’), and recommendations for 
improvement. Although researchers had the same protocol 
for all focus groups, the phase of AWSM implementation 
influenced which questions they asked at each group. For 
example, researchers did not ask about results from 
teachers’ use of formative assessment strategies in the first 
focus group because it was too early then for teachers to 
have substantial experience of using the strategies. 

Audio recordings of focus groups were transcribed, and 
the transcripts were imported into MAXQDA for analysis. 
Two coders developed initial codes based on the question 
topics in the focus group protocol. As coding progressed, 
the two coders conferred frequently on the coding scheme 
and the meaning of the codes, and collapsed some coding 
categories while expanding others into sub-codes. The final 
list of codes addressed session spacing and timing, fostering 
teacher collaboration, session mathematics content topics, 
facilitator support, understanding of formative assessment, 
need for professional development in formative assessment, 
implementation of formative assessment strategies, planning 
for formative assessment, formative assessment for 
differentiation, and results from formative assessment 
strategies. The emphasis of the analysis was to represent the 
points of view of the respondents, while also including some 
representative comments. 

After each iteration of AWSM, the evaluators analyzed 
focus groups feedback and provided the developers with 
formative information to revise the PD materials. For 
example, the final version of AWSM included more content 
information in the introductory summer session and an 
earlier introduction of the work sample rubric, in response 
to teacher feedback. 


Analysis and Results 

Results below are provided for the teacher work samples, 
teacher pedagogical content knowledge assessment, student 
achievement, and focus groups. 
Teacher Work Samples 

Table 1 displays descriptive statistics by work sample 
dimension as well test statistics and effect sizes for 
comparisons of pre- and post-test work sample scores. 
Teachers showed the greatest improvement in feedback: 
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Table 1 

Descriptive Statistics, Test Statistics, Effect Sizes for the Work Sample Pretest and Post-Test by Rubric Dimension (N = 47) 

Rubric Dimension Pretest Post-Test Test p-Value Cohen’s d 
Mean (SD) Mean (SD) Statistic 

Focus of the goals on student learning 22238270) 2, 32(.56) .80 43 45 

Alignment of learning goals and task 2tne(r2) 23250) Dene .03 16 

Alignment of learning goals and assessment criteria 1.38 (.49) 2.01 (.70) 5.02 .0001 1.03 

Clarity of the student assessment criteria 1.43 (.50) 2.24 (.70) 6.54 <.0001 Lao 

Feedback type 1.34 (.60) 2.53 (.60) 9.81 <.0001 1.98 

Feedback integrates student involvement Ledge (.38) 2.49 (.67) 12.48 <.0001 2.42 

Average score across all dimensions 1.63 (.54) 235606) ) 8.74 <.0001 72 





“feedback integrates student involvement” (d= 2.42) and 
“feedback type” (d = 1.98). Participants’ change scores also 
showed high effect sizes (greater than 1) on the two 
dimensions involving assessment criteria: “alignment of the 
learning goals and assessment criteria” (d = 1.04) and “clarity 
of the student assessment criteria” (d= 1.35). Participants 
showed the least improvement in the two dimensions 
involving learning goals: “focus of the goals on student 
learning” (d= .16) and “alignment of learning goals and 
task” (d= .45), as they scored relatively high at baseline. 
Scores did not differ significantly by teacher experience level. 

Tests of statistical significance were conducted using 
HLM to account for nesting of teachers within school. 
Seven separate models were estimated for each of the work 
sample dimensions as well as the average score across 
dimensions. For the average work sample scores, the 
analysis revealed a significant increase from pre- to post- 
test (see Figure 4). For the analyses of individual 
dimensions, dimensions representing assessment criteria 
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and feedback (Dimensions 3-6) showed statistically 
significant increases from pre- to post-test. For dimensions 
representing learning goals, change in “focus of learning 
goals on student learning” (Dimension 1) was statistically 
significant from pre- to post-test; however, change in 
“alignment of learning goals and task,” while in the positive 
direction, was not statistically significant (see Table 1 for 
test statistics and p-values for each comparison). Overall, 
these findings suggest that the AWSM_ professional 
development program increased teachers’ formative 
assessment practice, particularly in the areas of assessment 
criteria and feedback. 
Pedagogical Content Knowledge 

Baseline pedagogical content knowledge was to be used 
as a covariate in the student achievement model to assess 
the extent to which it affected the impact of the professional 
development on teacher practice and student achievement. 
Across both content areas, there was wide variability among 
teachers on assessment scores. Specifically, standardized 
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Figure 4. Average work sample scores as a function of formative assessment dimension and time (pretest to post-test; N = 47). 
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Figure 5. Adjusted means for mathematics achievement growth from fall pretest to spring outcome for treatment versus counterfactual groups. 


scores ranged from —1.52 to 2.12 on the PFA assessment 
(mean = .41, SD = .80) and from — 1.64 to 1.91 on the NCO 
assessment (mean = .31, SD = .87). A comparison of scores 
between the two subtests revealed no statistically significant 
difference between NCO and PFA. In order to compare 
scores from the AWSM field test sample to the normed LMT 
sample, one-sample t-tests were performed. Results indicate 
that the study sample scored significantly higher than the 
normed sample on both the NCO subtests, indicating that the 
study sample had greater pedagogical content knowledge at 
AWSM course entry than did the population of teachers who 
had previously taken the LMT. LMT scores did not 
significantly predict change in teacher work sample scores, 
indicating that prior mathematics pedagogical content 
knowledge was not a moderator of these effects. Therefore, 
AWSM improved teachers’ formative assessment practice 
regardless of their mathematics instruction entry skills. 
Student Achievement 

For proximal student achievement (PD year, 7 schools), 
results indicate that the counterfactual group scored higher 
than the treatment group at the fall pretest (p <.0001, 
Glass’s A = —.08) and at the spring outcome (p < .0001, 
Glass’s A= -—.07). However, the treatment x time 
interaction was not significant, suggesting that there was no 
difference in growth from fall pretest to spring post-test for 
students who were exposed to the AWSM PD program 
versus the previous year’s students of the same teachers 
who were not exposed to AWSM. 

For distal student achievement (1-year post training, 1 
school), there was no significant difference between the 
treatment and counterfactual groups at the fall pretest or the 
spring outcome. In addition, the treatment x time 
interaction was not significant (see Figure 5). 

Focus Groups 

In early focus groups, teachers reported that they clearly 

communicated learning goals and success criteria to 
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students, and used ungraded practice quizzes so students 
would understand how well they were mastering the 
learning goals. Teachers reported that this helped clarify 
their teaching and let students know what they should 
expect to learn. Further, having clear learning goals and 
success criteria facilitated communication with students and 
parents. Teachers said that formative assessment data helped 
them work with students at various levels of understanding, 
by changing the structure of the classroom so that students 
were working on different assignments based on their level 
of mastery of the learning goal. This helped each group 
focus on the next step in the learning progression. 

According to teachers, AWSM helped them realize that 
students should be involved in the formative assessment 
process to increase student understanding and 
accountability and enable them to be more independent 
learners. For example, teachers gave students success 
criteria to use in self-assessing their level of understanding 
relative to the learning goal. They also paired students for 
more formal peer assessment. This required teachers to 
teach students how to provide feedback to one another and 
to monitor the peer feedback process. As teachers had more 
practice with involving students in the formative 
assessment process, they reported more successes. 

In later focus groups, as AWSM progressed and teachers 
got specific suggestions in implementing formative 
assessment strategies, teachers at all schools began using 
the strategies more often and in a more efficient way. For 
example, teachers at all schools used more ungraded 
activities and found ways of providing feedback. They 
reported implementing low-risk ungraded quizzes as 
checkpoints; students working on these quizzes to get ready 
for the summative assessment could redo their work until 
they achieved mastery. In order to give feedback efficiently, 
one teacher arranged his/her classroom in centers to allow 
for more opportunities for one-on-one feedback. Another 
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teacher devised a system of two-letter codes that 
corresponded to items of feedback, such as “check to see if 
your answer is realistic (number sense).” This teacher 
explained, “You know, I’m trying to find as efficient a way 
as I can to make sure they’re not practicing doing it wrong.” 
One teacher commented that whenever he/she provided 
students with ungraded written feedback on homework 
assignments, students subsequently performed better on 
those topics in summative assessments. 

At every participating school, teachers reported that their 
students initially resisted ungraded practice and expected a 
grade for every activity. Some of these teachers said that 
they persisted with the technique until students came to 
understand the advantages it has for them, such as not being 
assigned a grade until they had had sufficient practice. 
Teachers also linked ungraded practice activities to 
supporting the effort, focus, and perseverance mathematical 
practices of the Common Core, and also told students that 
ungraded assignments helped teachers understand how 
students were progressing to the learning goal. 

At four schools, teachers revealed that they were doing 
learning targets differently than before, although these 
AWSM sessions had initially gotten the least positive 
teacher feedback. For example, one said, “I’ve always 
written my learning targets on the board, but now maybe 
they’re very specific. I make sure the kids are aware of what 
we’re doing today.” These teachers also took advantage of 
success criteria templates provided in AWSM to create 
written rubrics for their students so they could better 
understand the steps to mastery. One described the process: 
“T worked off of that [template] for weeks, developing more 
success criteria and going into a simple rubric .... And 
having that as an electronic file that I could tweak and turn 
and change for each one of my classes, that was a huge 
deal.” \ 

Teachers at five schools reported changing their feedback 
practices to avoid focusing primarily on getting right 
answers. They described their struggling students as 
initially reluctant to engage in complex problem solving out 
of fear of getting a wrong answer; these students would 
avoid the task or try to get teachers to simplify it or tell 
them exactly what to do. Several teachers would supply 
answers to problems so that students would focus instead 
on the process and be able to tell whether their answer was 
correct. One said, “I try to implement, through the year, I 
don’t care what the answer is. Show me how you get to the 
answer, all the steps. And it kind of puts them at ease a little 
bit that they...aren’t afraid to be wrong anymore, and 
hopefully it’ll help with that perseverance part.” 
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At five schools, teachers said they were using peer 
assessment extensively. They described having students 
partner to discuss their approaches to homework problems 
or to go over in-class activities. One commented that this 
technique made it easier for students to get help when they 
were reluctant to seek help from the teacher. Teachers 
found that it was important to model and demonstrate to 
students how to give effective peer feedback. For example: 


Well, I used the Two Stars and a Wish [a peer feedback 
organization strategy], and I just put that right on the 
paper so they kind of knew ahead of time that they were 
going to get peer feedback. And the first time we did it, 
I kind of gave an example and showed some things that 
I wanted. I gave some counterexamples like, “Don’t 
say, ‘Nice handwriting,’” or something like that. 


Teachers who used peer assessment sometimes found 
that students could better explain errors and next steps to 
peers. One said: 


When you walk around the room and you listen to their 
discussions, they’re really good discussions. Sometimes 
they can point something out to another student that’s at 
their level better than I can. Sometimes, we come in it 
from the teacher point of view, and the kid point of 
view they’re struggling along with them and they find 
ways to get through to them and communicate to them 
that we hadn’t thought about. 


Another teacher said, “I’ve had several kids, when 
working together, say ‘Oh, now I understand it because so 
and so explained it to me.’ And it’s like that’s good because 
sometimes they’ll say something a little bit differently, and 
it makes more sense.” Some of the teachers pointed out that 
effective peer feedback helps the student giving feedback as 
well as the one receiving it in any interaction. For example: 


[T]heir feedback was actually better than their actual 
work on the original assessment .... As far as the actual, 
the task itself, there were some kids who did really, really 
well and some kids not so much, but the actual feedback, 
even the kids who didn’t do so well on the [assignment], 
they started recognizing, “Oh yeah, that would have been 
a better approach” and so they actually. ..were learning 
even though they didn’t understand how to do it 
themselves initially. I think by them reading someone 
else’s they actually made sense of it. 
Some teachers mentioned that peer feedback helped 
students understand what teachers are looking for when 
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they ask students to show their work—the steps in the 
students’ approach to problems. One mentioned pairing 
students who show their work for peer assessment with 
those who generally do not, because peer feedback about 
how to show work was more effective at getting students to 
try it than was teacher encouragement. 

At two schools, teachers were initially concerned that 
students would be unkind to one another during peer 
feedback. This turned out to be a problem when peer 
feedback was given in written, anonymous form. However, 
when teachers then tried structured, face-to-face peer 
assessment, it worked well: “I find when they verbally 
feedback to their peer, they’re much nicer, it’s more 
constructive, and it’s actually a lot more helpful.” 

At three schools, teachers also focused on student self- 
assessment. They gave students tracking sheets of success 
criteria so they could understand their progress. One said, 
“They’ll start on what they know, but then they actually 
take ownership of saying, ‘Oh, I haven’t mastered this.’ 
And then they start testing their own learning.” Another 
teacher described how he/she used red, yellow, and green 
cards that the students would periodically hold up to 
communicate their understanding of the learning objective. 
This helped him/her know when to move on and when to 
spend more time on a concept. Regarding self-assessment 
strategies, one teacher said, “it’s also teaching them to be a 
little bit more independent workers, and be problem solvers 
on their own, and figuring it out.” Teachers at these three 
schools said they looked forward to planning for student 
self and peer assessment from the beginning next year to 
establish routines and help students recognize their progress 
toward mathematics learning goals. 


Discussion 

The AWSM professional development was feasible to 
implement in an actual school setting. The work samples 
helped to ground assessment discussions in teachers’ actual 
practice. The emphasis on formative assessment as a 
process including learning goals, success criteria, aligned 
tasks, and feedback encouraged teachers to see it as a way 
of framing instruction, rather than as something separate 
from the teaching/learning cycle. The developers made 
adjustments to the AWSM schedule and materials over the 
three iterations in order to respond to participants’ 
performance and suggestions. 

Effect sizes for changes in assessment work sample 
scores from pretest to post-test indicated that teachers 
overall demonstrated significant improvements in their 
formative assessment practices, with the greatest 
improvement in the two dimensions involving feedback 
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and on the two dimensions involving assessment criteria. 
Participants showed the least improvement in the two 
dimensions involving learning goals, in part because of 
relatively high baseline scores. The pretest of mathematics 
pedagogical content knowledge did not moderate the pre— 
post difference in teachers’ assessment skill. Overall, these 
findings suggest that the AWSM professional development 
program improved teachers’ formative assessment practice, 
particularly in the areas of assessment criteria and feedback, 
and that it did so regardless of teachers’ prior mathematics 
pedagogical content knowledge. 

Although literature on formative assessment documents 
learning gains associated with higher-quality practices (e.g., 
Black & Wiliam, 1998a, 1998b), in this study there was no 
statistically significant positive impact of AWSM on student 
achievement as measured by MAP during the training year, 
despite the change in teacher practice. MAP data on the 
post-training-year were available only for one school, and 
also did not show statistically significant impact. However, 
because AWSM was implemented over an entire school 
year and because it encouraged teachers to think of 
formative assessment as an approach to instructional 
practice affecting the whole teaching-learning cycle, it is 
unlikely that student impacts would be seen in a training 
year. Future research will focus on student achievement 
outcomes in the year prior to AWSM professional 
development implementation, when teachers can plan to use 
the AWSM strategies from the beginning of the year. 

In focus groups, participating teachers said that their 
students initially had the same problems reported in the 
literature with student motivation for learning mathematics, 
such as difficulty with engagement and persistence, 
especially with challenging problems. Their students also 
were reluctant to be wrong, to show work, and to do work 
that was ungraded. When AWSM teachers were able to 
involve students in understanding learning targets and in 
tracking progress on success criteria, they found that 
students were more likely to seek clarification of the goals 
and contribute to development of the success criteria. 
Teachers were able to group students to work together 
supportively, and to implement student peer assessment and 
self-assessment, even with struggling students. With the 
peer assessment process, using success criteria focused 
students on providing feedback relative to levels of 
performance, not just about whether the answer was right. 
Teachers were able to implement ungraded assignments by 
explaining to students their purpose, although some 
students struggled with this because students were used to 
receiving “points” as an indication of proficiency. 
However, when practice activities in the classroom were no 
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longer high-stakes and graded, students were more willing 
to engage and persist in more complex problem solving. 
Because of the importance of complex problem solving to 
the mathematical practices aspect of contemporary 
standards, and literature supporting the role of formative 
assessment in motivation and persistence, future research 
with AWSM will focus on measures of motivation in 
mathematics, and performance in solving complex 
problems (which was not addressed by the assessments we 
used as the achievement outcome in this study). 

Teachers were able through the AWSM sessions to share 
their assessment practices and learn from their peers. With 
peer support, they were able to adjust their mindsets about 
what students were able to do when they shifted from a 
teacher-centered classroom to a more student-centered 
approach. Finally, teachers recognized that implementing a 
formative assessment process did not mean adding another 
initiative to an already full plate of programs but was part of 
effective teaching and student learning. 
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Place value understanding requires the same activity that students use when developing fractional and algebraic 
reasoning, making this understanding foundational to mathematics learning. However, many students engage successfully 
in mathematics classrooms without having a conceptual understanding of place value, preventing them from accessing 
mathematics that is more sophisticated later. The purpose of this exploratory study is to investigate how upper elementary 
students’ unit coordination related to difficulties they experience when engaging in place value tasks. Understanding 
place value requires that students coordinate units recursively to construct multi-digit numbers from their single-digit 
number understandings through forms of unit development and strategic counting. Findings suggest that students 
identified as low-achieving were capable of only one or two levels of unit coordination. Furthermore, these students relied 
on inaccurate procedures to solve problems with millennial numbers. These findings indicate that upper elementary 
students identified as low-achieving are not to yet able to (de)compose numbers effectively, regroup tens and hundreds 
when operating on numbers, and transition between millennial numbers. Implications of this study suggest that curricula 
designers and statewide standards should adopt nuances in unit coordination when developing tasks that promote or 


assess students’ place value understanding. 


Place value understanding requires the same activity that 
students use when developing fractional and algebraic 


reasoning (Norton & Boyce, 2015), making this 
understanding foundational to mathematics learning. 
However, many students engage successfully in 


mathematics classrooms without having a_ conceptual 
understanding of place value, preventing them from 
accessing more sophisticated later. With that said, many of 
these students are still not successful on place-value related 
items in upper elementary. Four in seven questions on the 
2015 National Assessment of Education Progress (NAEP, 
2015) relate to place value understandings with only 40% of 
fourth-grade students performing at or above the proficient 
level (NAEP, various years 1992-2015). Wright, Ellemor- 
Collins, and Tabor (2012) have ‘investigated upper 
elementary students’ place value development and provided 
detailed means of instruction and assessment for low- 
achieving students. However, absent from their work are 
detailed stages of conceptual place value understanding 
illustrating degrees of underlying mental activity in low- 
achieving students. 

Battista (2012) argues that for an educator to “remediate 
a learning difficulty requires a more detailed picture” (p. 7), 
as educators are often not aware of students’ underlying 
mental activity. In this study, we frame this mental activity 
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with unit coordination literature (Norton & Boyce, 2015), 
to describe students’ construction of composite units 
(groups of items composed of more than one item) and their 
ability to coordinate these units when understanding place 
value. The purpose of this study was to investigate 
elementary low-achieving students’ unit coordination when 
engaging in place value tasks. We define a “low-achieving 
student” as a student achieving below standards established 
by research related assessments and tasks (Ginsburg & 
Baroody, 2003). Our inquiry focused on the following 
research questions: 

1. What types of unit development difficulties are prevalent 
in the place value understanding of these low-achieving 
students (Grades 3, 4, and 5)? 

2. How do these difficulties appear to manifest themselves 
as possibly limiting low-achieving students’ development of 
place value concepts? 

In this study, the authors adopt the constructivist 
paradigm, meaning that students learn mathematics through 
their active engagement with mathematics materials 
(Clements & Battista, 1990). We adopt this epistemological 
lens as informing our conceptual framework and review the 
literature on children’s grouping and counting actions for 
the construction of multi-digit number understanding. 
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_ Literature Review and Conceptual Framework 

Research shows that students are capable of constructing 
place value understandings earlier than previously thought 
(Resnick, 1984). Further, researchers report that children 
who can engage with multiple forms of activity (ie., 
counting objects, grouping, estimating) construct more 
robust multi-digit understandings (e.g., Chan, Au, & Tang, 
2014; Fuchs et al., 2010; Resnick, 1984). However, 
educational researchers do not yet know which of these 
forms of activity can best be leveraged to support low- 
achieving upper elementary students. In the following 
sections, we consider how unit coordination and conceptual 
place value relate to students’ place value development. 
Unit Coordination as a Conceptual Framework 

Place value is grounded in unit coordination. Students 
first construct number through a building up of empirical 
activity into an abstract series of coordinated actions 
whereby part-whole understandings are reorganized and 
extended to construct number as a mental object to act upon 
(Norton & Boyce, 2015; Steffe, 1994, 2001). The 
construction of mental objects forms the basis for unit 
construction and coordination, as students rely on concrete, 
pictorial, fingers, symbolic numerals, and language to 
evidence internalized (being able to mentally re-imagine 
contextual actions) or interiorized (being able to draw on 
de-contextualized actions) actions. 

When students solve problems with multi-digit numbers, 
they might draw upon forms of unit construction and 
coordination. Students understanding how proportional 
relationships grow by powers of 10, suggests that they 
would need to coordinate as many as four units (i.e., 10° X 
10' X 10° ...). For example, for students to conceptually 
understand 1,000, they would need to understand how it 
relates proportionally to 100 and 10,000 (107 x 10! and 
10*/10'). This requires multiplicative operations of 
exponential numbers, indicative of relatively more units to 
coordinate than if the student were asked to simply 
understand 1,000 in isolation (10°). Students unable to 
coordinate these exponential relationships might simply 
describe larger numbers (similar to what is illustrated above) 
as smaller numbers with several zeros and use language 
such as, “‘one with three zeros’ or “one-thousand” but not be 
able to represent the value of this number concretely. 
However, once number is interiorized it becomes a mental 
object to act upon. Norton and Boyce (2015) delineate 
stages of unit coordination as attributable to students’ 
number development (see Table 1). They explain that 
students capable of building a composite unit with a series 
of ones may be utilizing one level of unit coordination 
(Stage 1) to understand two levels of units. However, for 
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Table 1 
Three Stages of Unit Coordination 


Students’ Mental Actions 


Students rely on one level of units (i.e., units of 
“one”) and can build towards the coordination of 
two levels of units. 

Students rely on two levels of units (i.e., units of 
“five” as composed with “five units of one”) and 
can build toward the coordination of three levels 
of units. 

Students rely on three levels of units (i.e., units of 
“fifteen” as composed with “five units of three”). 


Stage 1 


Stage 2 


Stage 3 


Note. Table interpreted from Norton and Boyce’s (2015) 
Tablewiyp. 212. 


students to operate at Stage 2, they would need to iterate a 
composite unit (i.e., three is repeated five times) to build a 
new composite unit (i.e., fifteen). 

Finally, Norton and Boyce (2015) explain that students at 
Stage 3 are able to understand simultaneously how all three 
levels of units relate, as they have developed mental 
structures for number. For example, a student at a Stage 3 
level of understanding can understand how a unit of “one” 
relates to a composite unit of a composite unit (i.e., three 
sets of five). Students unable to coordinate all three levels of 
units simultaneously may build toward these mental 
structures through a “counting by” strategy or may count 
all. When developing Stage 3 unit coordination, students in 
early elementary grades are required to (1) develop mental 
activity necessary for conceptual place value development, 
and (2) develop units through counting and grouping. These 
two themes will be discussed more fully in this article. 
Mental Activity Necessary for Conceptual Place Value 
Development 

Procedural knowledge should be an extension of 
conceptual knowledge (Hiebert & Lefevre, 1986). Clark 
and Kamii (1996) found that first- through fifth-grade 
students were capable of constructing tens and ones for 
conceptual place value development. This finding seemed 
to occur when tasks afforded students opportunities to 
operate on multi-digit addition numbers and develop their 
own procedures for these operations. Thus, students were 
much more capable when they constructed procedures and 
relied on their internal mental activity. 

Battista (2012) delineated a set of learning progressions 
that explain mental activity that students’ rely on when 
developing conceptual place value. Seven levels delineate 
these actions and operations that ultimately describe 
students as beginning with concrete manipulative 
engagement, then transitioning toward abstract number 
engagement, and eventually developing algorithmic 
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procedures for multi-digit numbers (p. 10). The stance that 
Battista takes on his learning progressions is that educators 
need to transition from a deficit model toward a preventive 
model when assisting low-achieving students in their place 
value development. However, missing from Battista’s 
delineation of these levels is how to serve low-achieving 
students who are operating in schools that use intervention 
models. In other words, how can educators intervene to 
support low-achieving students who are already many 
school years behind in meeting statewide standards and 
classroom objectives? 

Ellemor-Collins and Wright (2011) studied conceptual 
place value development with low-achieving students in the 
upper elementary grades. Similar to Battista (2012), 
Ellemor-Collins and Wright found that students struggling 
with place value understanding should engage in tasks that 
extend from concrete to abstract material, and extend from 
place values of 200 and beyond. Furthermore, students 
capable of complex unit exchanges (ie., transitioning 
between ones to tens to hundreds back to ones) were more 
capable of extending number understanding beyond 1,000. 

Wright et al. (2012) delineate instruction for low- 
achieving students, and connect number “structures” to 
conceptual place value development. They define this 
structuring of numbers as students’ understanding that 
number is a structure and structured with smaller structures 
(similar to part-whole reasoning). Three sets of mental 
activity are required for structuring of numbers: (a) 
combinations and pattern development, (b) part-whole 
number construction, and (c) relational thinking. These are 
very similar to unit coordination. However, these types of 
mental activity do not explain the degree of unit coordination 
students may be engaging with as tasks change. 

In addition, Wright et al. (2012) posit that three 
dimensions should be used when supporting students’ 
conceptual place value.development. These dimensions 
include: (a) extending ranges of numbers, (b) making the 
increments and decrements of number more difficult, and (c) 
distancing the setting. Absent from this discussion are 
trajectories that explain specific degrees of change in mental 
activity when understanding place value. We believe that by 
adopting this unit coordination lens we can explain more 
detail regarding the degree of change in students’ responses. 

Structuring of numbers explains one form of mental 
activity students develop with conceptual place value. 
However, another form of mental activity that children 
develop with conceptual place value is a mental number line 
development (Dehaene, Izard, Spelke, & Pica, 2008; 
Moeller, Pixner, Kaufmann, & Nuerk, 2009; Siegler & 
Booth, 2004). Quite often mathematical tasks and 
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assessments that use a physical number line measure 
changes in students’ mental number lines. For instance, 
Siegler and Ramani (2008) found that when young children 
from low-socioeconomic status homes played games with 
linear number boards (similar to number lines) their 
estimation proficiency equaled that of their higher-achieving 
peers. Siegler and Booth (2004) also used a number line to 
measure first- and second-grade students’ understanding of 
number. The findings indicated that when these students 
used a number line there were significant correlations 
between their estimating and mathematical achievement 
(pr(15) = —.60, p<.01 for first-grade students and 
pr(17) = —.76, p < .01 for second-grade students). 

Many findings suggest that students in the early grades 
begin understanding magnitudes of number by relying on a 
mental logarithmic number line before developing a mental 
linear number line (Dehaene, 2011; Dehaene et al., 2008; 
Siegler & Booth, 2004). However, Moeller et al. (2009) 
found that children may be relying on two mental linear 
number lines (one for numbers 1—10 and one for one-digit 
and two-digit numbers 1-100). Moeller et al. (2009) explain 
why these findings conflict with Dehaene et al.’s (2008) 
logarithmic number line theory, as children in Moeller 
et al.’s study were shown Arabic numerals and children in 
Dehaene et al.’s study were verbally told numbers. Thus, it 
seems symbols and verbal number words may be processed 
quite differently by young children when constructing multi- 
digit numbers. These findings describe the comprehensive 
nature of children’s place value development. Children 
developing conceptual place value need to transition from 
concrete to abstract experiences, transition from low 
numbers to high numbers, rely upon a number as a structure, 
and rely upon their own mental number lines. Yet, with this 
comprehensive perspective on place value understanding, 
there is still much in the literature that is unknown. 

Unit Development Through Grouping and Counting 

Young children develop abstract units in upper 
elementary grades by engaging in physical grouping 
actions and spatial reasoning activities (Battista, 2004; 
Jones, Thornton, & Putt, 1994; Long & Kamii, 2001; 
McGuire & Kinzie, 2013; Reynolds & Wheatley, 1996; 
Winer & Battista, 2015; Wright et al., 2012). For the 
purpose of this study, we will focus mainly on relationships 
between grouping and unit coordination. McGuire and 
Kinzie (2013) found that when young children grouped 
objects in units of 10, they attended to groups of tens in 
numbers, but could not coordinate tens and leftovers. Fuson 
et al. (1997) found children constructed multi-digit number 
understandings through a coordination of decades and ones, 
whereby two-digit numbers were unitized by tens (i.e., 
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fifty) and by ones (i.e., three). Fuson et al. (1997) explain 
that children in their final stages of multi-digit number 
understanding need to understand numbers either by a 
sequence of tens and ones (i.e., ten, twenty, thirty, thirty- 
one, thirty-two) or by separate tens and ones (i.e., one ten, 
two tens, three tens, and one, two is 32). 

Strategic counting is one means in which grouping and 
counting have been found to be effective for developing 
place value understanding abstractly (Chan et al., 2014; 
Steffe & Cobb, 1988). Chan et al. (2014) defined strategic 
counting as students’ grouping of objects when counting. 
Essentially, for students to be successful in with place 
value, they need to build operations of numbers by first 
internalizing counting actions and their associated 
coordinated actions before interiorizing the operations as 
mental structures (Chan et al., 2014; Norton & Boyce, 
2015; Resnick, 1984; Steffe & Cobb, 1988). 

Steffe and Cobb (1988) initially proposed that students 
compose and coordinate units when counting. Through 
counting, students understand units as sets of ones prior to 
the construction of tens and ones. Essentially, beginning 
with a number-word sequence, children are able to attribute 
their number-word sequence to perceptual items before re- 
presenting them with fingers or imagined figurative material. 

Chan et al. (2014) found that first-grade students’ 
strategic counting abilities related to their multi-digit 
number understanding in second grade. Additionally, 
young children strategically counting in different bases had 
a wide variety of reasoning about place value number 
understandings (Slovin, 2011). This suggests that children 
need to do more than strategic counting to understand place 
value (2011). 

Summary 

Much of the research in mathematics education provides a 
multi-faceted perspective on how low-achieving students 
conceptualize place value. However, educational researchers 
do not yet empirically understand relationships between 
low-achieving students’ unit coordination and their place 
value development. 

Much of the place value learning research within the 
domain of students’ number understanding is in the lower 
elementary grades. Those who study upper elementary 
students’ place value development describe learning 
progressions, where “representations of learning rely on 
predetermined benchmarks for student achievement across 
grade levels” (Weber, Walkington, & McGalliard, 2015, p. 
254). The distinction here is that a learning trajectory explains 
“how students might engage with tasks, reflect on tasks, and 
develop knowledge through work on tasks” (Weber et al., 
2015, p. 254). In addition, by adopting unit coordination as a 
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theoretical model our findings can uniquely explain degrees of 
cognitive change with a new perspective on student learning. 


Methods 

This investigation was part of a larger study that used a 
sequential explanatory mixed-methods design to categorize 
student responses when understanding place value 
(Creswell, Plano Clark, Gutmann, & Hanson, 2003). The 
conceptual framework we described earlier grounded our 
research design, as we used student responses (qualitative 
data) to construct conceptual measures of student 
understanding (quantitative data). This article reports on the 
analysis of the qualitative data because nuances in student 
responses were analyzed with descriptive statistics. We 
considered the quantitative data as representing the 
frequency of student responses but not as an objective 
representation of a reality. 

Participants and Setting 

The participants in this study were upper elementary 
students referred for participation by their classroom teachers 
(in the school setting) or their parents (in the clinic setting) 
because they had experienced difficulties learning school 
mathematics. More specifically, teachers and parents deemed 
students as “low-achieving” after students failed to meet 
research-based standards at the same rate as their peers. For 
instance, students enrolled in third, fourth, or fifth grade were 
included in this study because research indicates that students 
in this age range should begin constructing rational number 
understandings. Researchers included three grade levels to 
capture the range and variance of student understandings that 
may occur across this age and grade span. 

None of the participants were receiving special education 
services for mathematics. A total of 124 participants (63 
male) from two western, rural school districts participated 
in the study. The group consisted of 41 third-grade (ages 8— 
9), 61 fourth-grade (ages 9-10), and 22 fifth-grade (ages 
10-11) students. We did not collect socioeconomic data on 
individual students; however, most students (93%) attended 
Title I schools, suggesting a high probability that some 
participants were from low-socioeconomic homes. 

Twenty-six classroom teachers and __ researchers 
administered diagnostic assessments with the participants in 
a one-to-one setting. In this study, we describe each teacher 
participating as a teacher-researcher, as they were fulfilling 
both roles as a teacher and as a researcher. The research 
team trained the teacher-researchers to conduct the interview 
assessments during a two-day training session to increase 
the likelihood of reliable administration of the interview 
procedures. During the training session, the 26 teacher- 
researchers learned to administer the assessment protocol 
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and they had multiple opportunities to practice the 
assessment procedures with feedback from the researchers. 
Teacher-researchers practiced, reflected, and discussed the 
logistics and decisions that would need to make when 
administering the assessment and categorizing students’ 
responses. The two-day training session increased the 
likelihood that all members of the data collection team were 
consistent in the administration of the assessment. 
Assessment Procedures and Analysis 

The data collected for this study is a subset of the pre- 
intervention data collected from the administration of the 
Place Value Iceberg Intervention Diagnostic Assessment 
(IIDA) to students scheduled to participate in intervention 
sessions (Westenskow, Moyer-Packenham, & Child, 2017). 
In contrast to other place value assessments, the IIDA was 
designed to be used by teachers when providing Tier II 
place value interventions (Assessments are available at: 
http://targetingmathinterventions.weebly.com). The IIDA 
tool explicitly illustrates which underlying understandings 
students successfully and unsuccessfully rely on during 
Tier I instruction. To develop IIDA assessment questions, 
the place value literature was analyzed and essential 
components of place value learning and the types of student 
misconceptions and errors were identified. During the pilot 
assessments, interviews were video recorded and responses 
were analyzed to identify categorizations of responses and 
to ensure that the questions adequately assessed 
understanding. Assessments were also piloted with teacher 
administrators to ensure that the categories of responses 
were easily understood and reliably identified by teachers. 
Three different grade levels of assessments were developed 
to reflect the progressions typically used in school 
instruction. Although all three levels assess the same place 
value components, the grade-level questions varied in 
complexity. For example, third-grade questions were 
limited to numbers one -thousand and less, fourth grade 
were limited to one million and less, and fifth grade to one 
billion and less. All students were tested only on their grade 
level and the results of further assessment to determine if 
the students had mastered low levels was not reported. 

This study used 13 IIDA questions which integrated the 
foundational activity (as described by Sarama & Clements, 
2009) and different dimensions (as described by Wright 
et al., 2012) so we could clearly see how students used 
previous knowledge when structuring numbers. Unique to 
the mathematics education field was the theoretical lens we 
adopted when analyzing student responses. 

The research team administered the Place Value 
assessment to 112 of the participants in their school of 
residence and to 12 participants in a university clinical 
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setting. Assessment completion typically required 30-45 
minutes per student. All assessments were conducted in a 
one-to-one setting. For each question, the person 
administering the test presented a card to the student and 
then read aloud the question for the card. Although students 
were encouraged to use mental math strategies, the 
interviewers provided the students with paper and pencil to 
use if needed. Students were asked to explain their thinking. 
When categorization of a response was not obvious, test 
administrators probed the students by asking clarifying 
questions. If the meaning of students’ verbal responses were 
unclear, the students were encouraged to write down their 
responses for further clarification. During the assessment the 
test administrator either recorded the category of the 
response or wrote a description of the response. Students’ 
written work was collected. Assessments conducted in the 
clinical sessions were videotaped, but assessments 
conducted in the school were not. Teacher-researchers 
administered the assessments in the morning and met in the 
afternoon with the researchers to discuss student responses. 
For the quantitative portion of the analysis, which 
investigated the prevalence of unit development difficulties, 
we calculated the percentage of students’ responses for each 
category. We then used these descriptive data to consider 
conceptually how students’ responses within each 
component (i.e., regrouping, counting) might relate to their 
ability to coordinate units when understanding place value. 
Using qualitative methods, we employed a thematic analysis 
(Patton, 1990) and open and axial coding (Strauss & 
Corbin, 1990) to categorize students’ responses for each of 
the question types. 


Results 

We identified assessment questions within six constructs 
of place value, which related to students’ use of unit 
coordination with place value: Counting, Groups of Ten, 
Decomposing, Regrouping, Position Relationships, and 
Comparing. We discuss these results in the sections below. 
Counting 

The Counting questions assessed students’ ability to count 
sequences and to count across transitions (see Figure 1). 


preoo. 
Begin with 570 at the red & % 
circle and countby 10sto oO O 
the green circle. ~ & 
O oO 


Figure 1. Example of counting questions. Adapted from Shumway (2011) 
number sense tasks. [Color figure can be viewed at wileyonlinelibrary.com] 
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Table 2 
Percent of Rubric Responses to Counting Questions 


Q 
= 
po 
Q. 
o 


Concept 


Counting backward from 106 to 92 
Counting backward from 106 to 92 
Counting backward from 106 to 92 
Counting by 10’s from 570 to 710 
Counting by 10’s from 970 to 1,110 
Counting by 10’s from 570 to 710 
Counting by 2’s from 888 to 916 
Counting by 2’s from 888 to 916 


MAWmnbBWnrh WwW 


Incorrect Correct 
Sequence Transition Pauses Fluent 
4.9 24.4 9.8 61.0 
10.0 oes wei 65.0 
0 19.0 4.8 On 
9.8 19.5 9.8 61.0 
15.0 61.7) Lh il ach 
oD 9.5 23.8 Shae 
Cet 39.0 Tee Sir 
28.6 23.8 14.3 3318 





Note. N = 3-41, 4-61, 5-22. 


We first examined students’ ability to sequence numbers. 
The results indicate that most student (more than 85%) 
correctly sequenced numbers when counting backwards by 
ones and forwards by tens (see Table 2). However, a higher 
number of students (17.1—-28.6%) struggled when counting 
by twos. An explanation for this difference may be that 
both counting by tens and counting backwards are done in 
increments of one in the tens and ones positions, while 
counting by twos requires students to think in increments of 
two. 

For students who correctly counted in sequence, we next 
examined their counting across century transitions. 
Counting across century transitions requires the 
coordination of at least two units because the number to 
count on from (e.g., 99, 199) needs to be reorganized with 
the new century (e.g., 100, 200). We identified three 
categories of transition responses; Incorrect; Transitions 
with Long Pauses (three seconds or more), and Fluent 
transitions. Incorrect century transition errors occurred 
when students stopped counting at the century, substituted a 
different century, or incorrectly skipped to an incorrect 
millennial position. 

As shown in Table 2, the analysis suggested two trends: 
First, students tended to make more transition errors when 
counting by twos than when counting backward or by tens. 
This may be due to the need to coordinate both groups of 
two and the regrouping of the tens unit when making a 
century transition. Second, when asked to count across the 
one thousands position, 61.5% of fourth-grade students 
made a millennial transition error (see Table 2). This action 
required the coordination of three units as students had to 
count by 1 and reorganize the ten’s and hundred’s places. 
Thus, students’ coordination of units may develop slowly 
from grade level to grade level, but the tasks also develop 
requiring more sophisticated coordination with higher 
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composite units (millennial transitions) and decomposing 
(backward counting). 
Groups of Ten 

We used two questions to assess students’ use of groups 
of ten when interpreting numbers (see Figure 1). We 
identified four categories of responses for question 1; No 
Grouping, Partial Grouping, Immediate Grouping, and 
Unitizing (see Figure 2 and Table 3). 

Students in the No Grouping category (18.3-24.4%) 
counted the hearts one-by-one, suggesting that students 
were at Stage 1 unit coordination (i.e., not acting upon 
groups of ten as an object) Students in the Partial 
Grouping category, (0—21.7%) counted the first two 
columns by ones and then used groups of ten to finish 
counting. Students in the Immediate Grouping category 
(23.3-47.6%) counted one group of ten and then 
counted the remaining groups by tens. These two types 
of responses suggest that students were at Stage 1 unit 
coordination, but using the material effectively to build 
toward coordinating two sets of units. Students in the 
Unitizing category (28.6-36.7%) answered 42 
immediately. Their explanations suggested that they 
anticipated and then subitized the groups of ten. This 
response suggests that students were at Stage 2 in their 
use of unit coordination. 


1. How many hearts 
are in this picture? 


2. This picture has 35 hearts. 
Cirle how many hearts the or eewwen 
five represents. Circle how VyVVVyyY 
PO Ooo 
many hearts the three Sve eg 
represents. Pe ae 


Figure 2. Groups of ten questions. [Color figure can be viewed at 
wileyonlinelibrary.com] 
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Table 3 
Percent of Students in Each Group of Ten Category 





Rubric Percent 











Grade Concept No Grouping Partial Grouping Immediate Grouping Unitizing 
3 42 hearts grouped in 10’s 24.4 Dee ene Shes 
4 42 hearts grouped in 10’s 18.3 7 23.3 36.7 
5 42 hearts grouped in 10’s 23.8 0 47.6 28.6 
Note. N = 3-41, 4-61, 5-22. 
Table 4 Decomposing 
Percent of Rubric Responses to the 35 Hearts Question To assess decomposing, we used an addition and a 
Percent Percent multiplication question (see Figure 3). The questions 
Incorrect Correct assessed students’ use of Stage 2 unit coordination. 
Giads Concept NO Place Cotints "" Gnitizing Through these tasks students were required to act upon 
Value 30 units of ten by partitioning composite units and then 
disembedding these composite units with their 
3 Model 3 and 5 in 35 39.0 ie 43.9 . 
4 Medeli3‘and:5 i135 32.8 263 410 complements of ten. Students need to coordinate at least 
5 Model 3 and 5 in 35 61.9 0 38.1 two units to be successful in the addition task and possibly 


Note. N = 3-41, 4-61, 5-22. 


The second Groups of Ten question assessed students’ 
symbolic understanding that two digit numbers consist of 
groups of ten and single units. Three categories of 
responses were identified; No Place Value, Counts 30, and 
Unitizing) (see Table 4). Student responses in the No Place 
Value category circled five hearts and then three more 
hearts. (32.8-61.9%). This response did not reflect an 
understanding of the difference between the place value 
positions. This response also suggests students were not 
coordinating place value units, indicative of Stage 1 unit 
coordination. In contrast, student responses in the other two 
categories did reflect the place value positions as students 
circled five hearts and then circled 30 hearts. The difference 
between the two categories was that students in the Counts 
30 category (0-26.3%) counted and then circled 30 hearts 
while students in the Unitizing category circled the 
remaining hearts without counting (see Table 4), The 
Counts 30 category is indicative of Stage 1 coordination 
whereas the Unitizing category is indicative of Stage 2 unit 
coordination. 


Using addition write as many Se es 
combinations making 120 as you 
can. Reversing numbers such as —— 


2+ tand 1+ 2 count as the 
same combination. 


Figure 3. Examples of the addition and multiplication decomposing questions. 
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= 120 
+ = 120 


+ = 120 


three units to be successful in the multiplication task. 

These two questions revealed the degree of difficulty 
many students had when decomposing numbers. For the 
third-, fourth-, and fifth-grade students 12.2, 18.0, and 
18.2%, respectively, percent of the students did not write a 
correct addition combination (see Table 5). The percent of 
third-, fourth-, and fifth-grade students who identified three 
combinations were 56.1, 42.7, and 45.5%, respectively. 
The most common combination written was 100 + 20. 

For decomposing using multiplication, the percent of 
third-, fourth-, and fifth-grade students who did not write a 
correct combination (120 < 1 or 60 X 1 was accepted) 
were 22.0, 14.8, and 36.4%, respectively, and the number 
of students who wrote four combinations was 4.9, 9.8, and 
0%, respectively. An important difference between 
students’ accuracy when decomposing additively versus 
multiplicatively was students often additively compensated 
with a “+ 1” or a “+ 2” strategy rather than using groups 
of 10. The need to use multiplicative decomposing 
prevented students from using this compensation strategy 
when decomposing multiplicatively. Further, students 
responding in this way were at a Stage 1 or 2 unit 
coordination because they could coordinate one or two 


: Bee : x = 120 
Using multiplication write as =~ =~ 
many combinations making x =120 
120 as you can. Reversing —_ —— 
the numbers such as 1 x 2 x =120 
and 2 x 1 count as the aa 
same combination. x =120 
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Table 5 


Percent of the Number of Responses for Each Composing/Decomposing Question 


Number of Combinations 





Grade Question 0 1 Z 3 

3 Addition combinations to 120 12.2 29.3 2.4 56.1 
4 Addition combinations to 120 18.0 24.6 14.8 427 
5 Addition combinations to 120 18.2 Za 13.6 45.5 


Number of Combinations 





0 1 2 3 4 
5 Multiplication combinations to 60 22.0 24.4 41.5 Wes 4.9 
4 Multiplication combinations to 120 14.8 26,2 26.2 230 9.8 
5 Multiplication combinations to 120 36.4 50.0 13.6 0 0 


Note. N = 3-41, 4-61, 5-22. 


units, but struggled to multiplicatively understand the 
operative structure of unit needed for multiplication. 
Regrouping 

Regrouping requires students to act upon their counting 
and place value construction to regroup numbers. The three 
skills relevant to unit coordination assessed in this 
component were addition regrouping, subtraction 
regrouping, and adding/subtracting one at the transitions to 
tens and thousands (see Figure 4). 

For the regrouping questions we identified three types of 
regrouping responses: No Regrouping, Incorrect Procedure, 
and Correct Regrouping (see Table 6). In the first level, No 
Regrouping, students did not attempt to regroup. For 
example, to name 2 hundreds, 13 tens, and 1 ones, some 
students wrote 2,131 and when subtracting, some students 
subtracted the digit of the least value from the digit of the 
greatest value. These responses suggest that students were 
at Stage 1 unit coordination, as they did not appear to 
understand the conceptual need for a reorganization of units 
when operating on multi-digit numbers. 

In the Incorrect Procedure category, students attempted 
to regroup but made procedural errors, such as adding 10 
instead of 1 to the next position when adding or not 
exchanging ten rods for one hundred (fifth grade) or ten 
blocks for one ten rod (third, fourth, and fifth grades). 
These responses suggest that students were at Stage | unit 
coordination because students had procedures not grounded 


With base ten blocks, this little block 
represents one, these rods are ten 
blocks and the flats are 100 blocks. 
What number is being modeled by 
these blocks? 


Figure 4. Example of an addition and a subtraction regrouping question. 
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in conceptual understandings of number. In the Correct 
category, 85.4% of third graders correctly regrouped ones 
in a two-digit number, and 39.3% of fourth graders and 
54.5% of fifth graders correctly regrouped tens in a three- 
digit number when naming the modeled numbers (see 
Table 6). Less than one-half of the students correctly 
modeled regrouping when subtracting. These responses 
suggest Stage 2 or 3 unit coordination, as students were 
either building three levels of units or utilizing three levels 
of units to coordinate trading and (de)composition of 
number. 

The Before and After regrouping questions assessed 
students’ ability to regroup when crossing transitions (see 
Figure 5). These types of questions required students to 
engage in Stage 2 or 3 unit coordination. 

Analysis of the Before and After Question results yielded 
four categories of responses; Last Digit, Both Incorrect, 
One Correct, and Both Correct (see Table 7). For the Last 
Digit category, students added and subtracted one to and 
one from the last non-zero digit in the number. For 
example, students would respond that 102,000 and 104,000 
were one less and one more than 103,000. This suggests 
that students were not coordinating units of one thousand, 
but were focused on counting in sequence without 
consideration of the place value unit of the non-zero digit. 

In the second category, students incorrectly attempted 
regrouping. For example, some student responses to one 


Use these base ten blocks 

to make the number 262. 
Using the blocks show how 
to model the subtraction of 63. 


262-53 
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Table 6 


Percent of the Number of Responses for Each No Regrouping/Incorrect 
Procedure/Correct Procedure Question 


Grade Concept No Incorrect Correct 
Regrouping Procedure 

3 Regroup 5 tens 12 ones tS To 85.4 

4 Regroup 2 hundreds 29:5 ah) oa 
13 tens 1 one 

5 Regroup 2 hundreds 18.2 eles 54.5 
13 tens 1 one 

B 81-3 4.9 46.3 48.8 

4 262-53 aS Daud 44,3 

5 632-253 9.1 68.1 Zed. 


Note. N = 3-41, 4-61, 5-22. 


less than 103,000 were 103,999; 103,009; or 193,999. In 
the third category, One Correct, students answered one 
question correctly which typically was the “one after 
question,” suggesting that students could operate at Stage 1 
or 2 unit coordination. Students would need to have 
an operative structure for addition and _ subtraction 
relationships to be successful with the “one less” question, 
indicative of level three unit coordination. In the Both 
Correct level, there was a large contrast in the number of 
third-grade students (41.5%) and the number of fourth 
(12.3%) and fifth (14.3%) grade students who answered 
both before and after correctly (see Table 7). These findings 
suggest that students may have operative procedures for 
lower digit numbers, but struggle with recursive operative 
relationships when extending these procedures to higher 
digit numbers. 
Relationships 

Relationship questions assessed students’ conceptual 
understanding that each place value position is ten times the 
position to the right and 1/10 of the position to the left (a 
Stage 3 unit coordination requirement). Three types of 
questions were used to evaluate students’ understanding; 
(a) How Many Times Larger; (b) Multiplying by Multiples 
of Ten; and (c) Dividing by Multiples of Ten. At the 
beginning of the relationship component of the assessment, 
test administrators asked students how many times larger 
200 is than 20. In response, 34.1% of third graders, 37.7% 
of fourth graders and 22.7% of fifth graders responded with 
10 times larger. The other two groups of Relationship 
questions asked students’ to mentally calculate the solution 
to four multiple of ten multiplication and four multiple of 


If you were counting what number 
would come one before 203,000. What 
number would come one after 203,000 


203,000 


Figure 5. Example of before and after question. 
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Table 7 

Percent of Rubric Responses to Before and After Regrouping Question 
Grade Question Last Both One Both 

Digit Incorrect Correct Correct 

3 500 24.4 1794 way 41.5 
3 770 24.4 eo 13 61.0 
+ 203,000 49.2 131 16.4 orb 
5 103,000 28.6 he 61.9 0 

3 1,030,000 19.0 33.3 S30 14.3 


Note. N = 3-41, 4-61, 5-22. 


ten division expressions of increasing complexity (see 
Figure 6). 

Most students (90% or higher) correctly multiplied at 
least one of the four multiplying questions (see Table 8). 
Typically students correctly answered 3 X 10, a fact which 
is often memorized as part of multiplication fact practice. 
However, less than half of the students were able to 
correctly solve all four multiplication/division expressions. 
When asked to explain their strategies, many students did 
not refer to place value relationships but only to adding or 
subtracting zeros. It is probable that students were 
following procedural rules and were not engaging in unit 
coordination. This lack of unit coordination is further 
supported by the low number of students who responded 
that 200 was 10 times greater than 20. Students did not 
appear to understand or use the concept that a place value 
position is 1/10 of the position value to the left and 10 times 
the position value to the right. 

Most students tended to correctly solve more 
multiplication than division expressions. This suggests that 
some students struggled with the inverse relationship 
between multiplication and division and is indicative of 
students operating at Stage 2 unit coordination. 

Comparing Numbers 

The Comparing Numbers question relevant to unit 
coordination assessed students’ understanding of number 
density by asking students to identify a number between 
two given numbers (see Figure 7). Correctly naming 
numbers between two or three digit numbers required 
students to coordinate two units. Correctly naming numbers 
between numbers in the thousands and millions required 
students to coordinate three units. Students needed to use 
the relationships established when coordinating two units to 
understand the abstract relationships when coordinating 
three units. 

This analysis revealed four categories of responses: 
Correct; No number; Last Non-zero Digit; and Others. For 
smaller numbers such as between 30 and 40, and between 
600 and 700, 73.2 and 70.6%, respectively, of the third- 


25 


Components of Place Value Understanding 


3x1i0= 


What is the solution for 4x30 
each of the expressions’? 
iSxid= 
40x10=__ 


70 + 10= 
What is the solution for 400+10=_ 
each of the expressions? mise 


80=20= 


Figure 6. Examples of multiplying by multiples of ten and dividing by multiples of ten questions. 


grade students correctly identified a number (see Table 9). 
However, for larger numbers in the thousands and millions 
periods, less students correctly identified numbers between 
the two given numbers. This change in accuracy may be due 
to students’ inability to operate at Stage 3 unit coordination. 
Operating with larger and less familiar numbers requires 
more abstract, recursive, unit structure understandings. 

Students who gave No Number category responses 
typically first hesitated and then stated there were no 
numbers between the two given numbers. Students who 
gave Last Non-zero Digit category responses added or 
subtracted one to the last non-zero digit of one of the 
numbers (e.g., 412,000 is between 410,000 and 411,000). 
These two types of responses suggest that students were 
relying on sequence counting and were not coordinating the 
place value units. Students who gave Other category 
responses named numbers that were outside the range of 
the two numbers given and were not of the Last Non-zero 
Digit category (e.g., 411,001 or 499,000 for numbers 
between 410,000 and 411,000). 


Discussion 

In this paper, we examined elementary students’ unit 
coordination when engaging in tasks to explain Place Value 
understandings. In this section, we discuss the difficulties 
students exhibited with place value concepts and the 
educational implications of our findings. 
Difficulties Children Experienced When 
Understanding Place Value Concepts 

The third-, fourth-, and fifth-grade students in this study 
struggled with foundational aspects of place value 


Table 8 


relationships within multi-digit numbers. This was 
associated with their inability to flexibly operate on 
numbers and use their knowledge of place value between 
millennial numbers. Wright et al. (2012) posit that students 
need to engage with hundreds materials when extending 
their understanding to 1000. However, in our tasks we only 
gave the students in this particular study symbolic material; 
this prevented them from be able to make connections 
between concrete and symbolic material when engaging 
with millennials. Students were also unable to treat digits in 
their respective place value positions as anything more than 
the value of the digits themselves when regrouping and 
unitizing (i.e., the “3” in 34 is treated as a 3, not a 30). 
These findings are not new as they are very similar to the 
findings from Fuson et al. (1997) wherein students 
struggled when transitioning from single-unitary digit 
numbers to multi-digit numbers. These findings suggest 
that students in this study may have been low-achieving 
because they were not yet able to interiorize concepts of ten 
(Fuson et al. 1997). In addition, we posit that students were 
unable to coordinate three levels of units and may have 
benefitted from more opportunities to use concrete material 
to build toward three levels. The Fuson et al. (1997) task 
may explain when students are (un)able to coordinate two 
levels of units because they are not given concrete material 
to build toward two levels of unit coordination. For 
instance, because of the lack of concrete experiences, 
students may not be able to develop and use composite 
units as mathematical objects when operating on smaller 
and larger numbers, relatively. This aligns with Norton and 
Boyce’s (2015) discussion on unit coordination, which 


Percent of Students Who Correctly Answered None, One, Two, Three, and Four of the Multiplying/Dividing Questions 


Grade Concept 

310, 4 x 100, 12 X 100 and 60 X 100 
30/3, 40/10, 420/10, 400/100 

3 X 10, 4 x 30, 15 X 10, and 40 x 10 
70/10, 400/10, 700/70, 80/20 

3 X 10,4 X 1,000, 23 x 10 and 40 X 100 
100/10, 6,000/10, 1200/40, 25/100 


AMmn BBW WwW 


Rubric Percent 
hes Gt ee ee eee, 


None One Two Three Four 
aS 2.4 Baal 41.5 14.6 

17.5 75 B25 20.0 12.5 
1.6 4.9 Sila 18.0 44.3 
0) Desk 10.3 25.6 41.0 
4.5 0.0 225) 31.8 40.9 

36.4 1350 36.4 13.6 0 


ee ee eee 


Note. N = 3-41, 4-61, 5-22. 
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Name a number that would 
come between 410,000 and 
411,000. 


410,000 __ 411,000 


Figure 7, Example of a density question. 


suggests that students at the Stage 1 unit coordination 
understanding are not capable of developing multi-digit or 
composite units absent from concrete material and that this 
development only allows them access to tasks that require 
two levels of unit coordination, not three levels of unit 
coordination. 

Our findings also suggest that students must understand 
the relationships between column values (density), What 
this means for students is that they should understand how 
one place value column is one multiple of ten difference 
from a subsequent place value column. For students to 
understand and operate on multi-digit numbers, their 
understanding of the operational structures for multi-digit 
numbers must be recursive. Essentially, students should be 
able to extend their number understanding to larger 
numbers by incorporating operational structures of numbers 
developed through their prior unit coordination (Norton & 
Boyce, 2015; Wright et al., 2012). Students in this study 
simply manipulated the place values by adding or taking 
away zeros without understanding the changes in value that 
these actions produced. Wright et al. (2012) posit that this 
type of activity is what students and teachers expect when 
engaging with conventional place value instead of 
conceptual place value. We further argue that it is far more 
effective to understand how this conventional activity 
illustrates stages in students’ unit coordination. For 
instance, when engaging in this task students were typically 
operating at a Stage 1 unit coordination, preventing them 
from accessing Stage 3. If students were able to operate at a 
Stage 2 they did not always have strategies (i.e., counting 
by multiples of ten) to’build toward Stage 3. Students’ 
inability to have an interiorized structure of the proportional 
relationships may explain why they had difficulty 


understanding and operations to six- or seven-digit number 
understanding and operations. 
Counting, Grouping, and Unit Coordination 

We analyzed the data through the theoretical lens of 
students’ unit coordination abilities (Norton & Boyce, 
2015). Students were successful when engaging in tasks 
when characteristics of the tasks only required students to 
coordinate units at Stage 1 or 2. This was evident when 
students used groups to build toward three tens or “3” in 35, 
but they were not able to immediately draw on different 
groups of tens when (de)composing number. Also, students’ 
ability to find numbers in between two multi-digit numbers 
seemed to require two or three levels of interiorized units. 

This may explain why students in third grade were able 
to utilize a strategy when finding numbers in between two 
and three-digit numbers, but almost half of the students in 
fourth and fifth grade struggled to name a number in 
between six and nine digits. These findings suggest that 
students may be internalizing units to coordinate each time, 
but are not interiorizing recursive relationships of multi- 
digit number understandings as a mental structure. Wright 
et al. (2012) suggest that when students are developing 
place value they need to engage in the making of complex 
sets of increments and decrements with multiples of ones, 
tens, and hundreds. These actions further press students’ 
development of larger numbers later. In this study, we 
could argue that in engaging in these increments and 
decrements, students are constructing operations on and 
around numbers, which promotes more sophisticated unit 
coordination. 

It was also evident that, for students to provide multiple 
combinations for number and to develop operations for 
those combinations, students would need to coordinate 
more groups and develop more flexible, sophisticated part- 
whole understandings. For instance, when students were 
asked to describe as many as four multiplicative 
combinations for a number, they were required to 
coordinate units at stages 2 and 3, but when students were 


transitioning from two- and _ three-digit number asked to simply solve multiplication problems, many of 
Table 9 
Percent of Rubric Responses to Density Questions 

Incorrect 
Grade Question No Number Last Non-Zero Digit Other Correct 
3 Between 30 and 40 7.3 17.1 2.4 7By2 
3 Between 600 and 700 122 14.6 2.4 70.6 
4 Between 410,000 and 411,000 48.3 1.6 9.8 44.3 
5 Between 610,000,000 and 611,000,000 40.9 13.6 0 68.2 





Note. N = 3-41, 4-61, 5-22. 
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their responses required them to coordinate units only at 
Stage 2. This finding aligns with that of Wright et al. 
(2012), who reported that the more number structures 
children create, the more sophisticated their understanding 
for that number. 

Finally, findings from the regrouping and grouping while 
counting tasks suggest that the degree of the number of 
units needed to coordinate and the material provided to the 
students may (dis)allow them opportunities to build from 
one unit coordination stage to the next. For instance, in the 
regrouping tasks more than 85% of the third-grade students 
were successful when using a picture and two numbers to 
regroup. However, when the fourth- and _ fifth-grade 
students were given the same task, but with three numbers 
to regroup, only about half of the students were correct. 
Also, when third and fifth graders were asked to count by 
“two’s” their accuracy rates were much lower than their 
fourth-grade student counterparts who were asked to count 
by “10’s.” These findings suggest that unit coordination 
became quite difficult for three reasons: (a) more numbers 
to coordinate required more abstract unit coordination, (b) 
novel units to coordinate required more flexible unit 
coordination, and (c) images of blocks disallowed active 
engagement for some students when building from one 
stage of unit coordination to another stage. 

These findings suggest that when students in third grade 
were successful it was because they were given tasks that 
only required them to engage in Stage 2 unit coordination. 
Whereas fourth- and fifth-grade students were less 
successful when given relatively more sophisticated 
numbers, more units, and fewer pictorial items. These 
changes in assessment questions were fundamental in how 
students were able to access their counting, grouping, and 
unit coordination and suggest that students at different 
grade levels require different stages of unit coordination 
when engaging in place value tasks. 

Educational Implications 

These results have implications regarding the design of 
assessments and interventions for students’ understanding 
of place value. For instance, it is still not clear how 
educators can vary their instruction to address specific 
underlying mental actions that low-achieving students may 
require who are already many school years behind in 
meeting statewide standards and classroom objectives. 
Findings from this study do not directly address these 
concerns, but suggest that curricula and statewide content 
standards do not utilize important mental activity (e.g., 
Common Core State Standards for Mathematical Practices, 
2009; Principles to Action, 2014) necessary for low- 
achieving students when designing interventions. The 
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Common Core Mathematical practices that illustrate 
effective student engagement is not nuanced enough to 
describe how student engagement may appear when 
students are performing well below grade level and, more 
specifically, how this activity should differ for conceptual 
place value instruction. Thus, addressing these specific 
concerns would benefit these students with this particular 
form of number development. Assessment and intervention 
should focus on inferred mental actions that students rely 
on when solving problems and not just accurate procedures. 
Quite often statewide standards guide students’ place value 
assessments rather than engaging students in the necessary 
mental activity needed for success for these problems. 
Curricula designers and those designing statewide 
standards should consider these necessary mental actions 
when designing assessments and interventions. Many U.S. 
students without conceptual place value understandings are 
unable to reach Stage 3 unit coordination to successfully 
meet place value standards. This suggests that common 
standards and assessments need to change to reflect mental 
activity necessary for students’ conceptual place value 
understandings. 

More specifically, Wright et al. (2012) address the 
“structuring” of numbers students need to engage in when 
developing early number and conceptual place value 
understandings. Many of the interventions Wright et al. 
posit for place value development address activity with 
materials (e.g., incrementing and decrementing) and the 
proximity students have with materials (e.g., concrete, 
visualized concrete, verbal). Findings from this study echo 
these intervention models and explain the stage of unit 
coordination, which may be used by students when relying 
on activity and materials. Thus, we argue that curriculum 
should take on these aspects of of activity and materials 
while addressing the stages of unit coordination. 

A final educational implication is that curricula designers 
should design intervention procedures that provide more 
nuanced options that closely align with student’s place 
value understandings and abilities. Low-achieving students, 
are not yet able to master particular conceptual place value 
knowledge in lower grade levels, which then further 
distances them from achieving at the same rate as their 
peers in number and operations in upper grade levels. Thus, 
future research should use unit coordination to more fully 
develop learning trajectories that can support this curricula 
design. This new theoretical perspective can provide 
opportunities for educational stakeholders to consider the 
degrees of learning necessary for such a multi-faceted 
concept in mathematics. 
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This research addresses the issue of how to support students’ representational fluency—the ability to create, move 
within, translate across, and derive meaning from external representations of mathematical ideas. The context of solving 
linear equations in a combined computer algebra system (CAS) and paper-and-pencil classroom environment is targeted 
as a rich and pressing context to study this issue. We report results of a collaborative teaching experiment in which we 
designed for and tested a functions approach to solving equations with ninth-grade algebra students, and link to results of 
semi-structured interviews with students before and after the experiment. Results of analyzing the five-week experiment 
include instructional supports for students’ representational fluency in solving linear equations: (a) sequencing the use of 
graphs, tables, and CAS feedback prior to formal symbolic transpositions, (b) connecting solutions to equations across 
representations, and (c) encouraging understanding of equations as equivalence relations that are sometimes, always, or 
never true. While some students’ change in sophistication of representational fluency helps substantiate the productive 
nature of these supports, other students’ persistent struggles raise questions of how to address the diverse needs of 


learners in complex learning environments involving multiple tool-based representations. 


Developing students’ representational fluency—the ability 
to create, move within, translate across, and derive meaning 
from external representations of mathematical ideas (cf. 
Zbiek, Heid, Blume, & Dick, 2007)}—has remained a 
prevalent value in school mathematics for over two decades. 
Representational fluency is the linchpin in the National 
Research Council’s (NRC) (2001) description of conceptual 
understanding, and is increasingly important in technology 
rich school algebra (Heid & Blume, 2008). Indeed, 
representing and learning are linked as mutually supportive 
processes (e.g., see Selling, 2015). The purpose of this 
research is to explore links between instructional supports 
and students’ representational fluency in solving problems 
with computer algebra systems (CASs)/paper-and-pencil. 


Representational Fluency in Solving Equations 

Within the landscape of school algebra reform, both the 
content of a functions approach to solving equations, and 
the mathematical practice of reasoning fluently with 
multiple representations in technology rich settings are 
clearly valued (Arbaugh et al., 2010). The context of a 
functions approach to solving equations with CASs 
and paper-and-pencil is a rich context for exploring 
students’ representational fluency and supports for that 
development with secondary algebra students (Kieran & 
Yerushalmy, 2004), especially when CASs are treated as a 
representational toolkit to create and move between 
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representations (cf. Dick & Edwards, 2008). By a functions 
approach to solving linear equations, we mean that “a linear 
equation in one variable is, in essence, a specific instance of 
a corresponding linear relationship (or equation in two 
variables)” (Cai, Nie, & Moyer, 2010, p. 178) and includes 
comparing relationships among graphic, numeric, and 
symbolic representations of functions created with 
technology (Cai, Nie, & Moyer, 2010, p. 178). 

In research with graphing calculators, Kieran and Sfard 
(1999) investigate supports for students’ meaningful 
manipulation of equivalent expressions and Yerushalmy 
and Gilead (1997) investigate supports for students’ 
meaningful equation solving. Both studies suggest a 
teaching sequence by which graphical and numeric 
function table representations are introduced prior to formal 
symbolic representations. Yerushalmy and Gilead contend 
that once solving with graphs and tables, the students 
realized the need for more formal, exact methods of 
solving. Across these studies the authors report evidence of 
students’ meaningful manipulation of symbols, supporting 
the need to test such instructional supports with CAS. 

In contemporary research involving CAS, effects of 
technology-intensive instruction that emphasize a functions 
approach to solving linear equations lead to somewhat 
mixed findings on whether and how a functions approach 
supports students’ representational fluency in solving 
equations. In research by Hong and colleagues (Hong & 
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Thomas, 2002; Hong, Thomas, & Kwon, 2000) they found 
mixed effects of their instructional approach that showed 
students how to solve equations from an algebraic, 
graphical, and tabular perspective with the aid of the CAS. 
No statistically significant gains were made on measures of 
routine problems, yet gains on some problems involving 
solving across representation types were found. Other 
research has made progress in characterizing students’ 
representational fluency and problem solving (eg., 
Adu-Gyamfi, Bossé, & Chandler, 2015; Adu-Gyamfi, 
Bossé, & Stiff, 2012), yet relatively few studies link 
representational fluency and instruction (see Selling, 2015 
as a notable exception). 

To build on this work, there is a need for greater attention 
to clarify how functions approaches are leveraged in 
CAS/paper-and-pencil environments, including how 
representations are sequenced and connected, and how those 
supports are linked to students’ change in representational 
fluency. In a CAS/paper-and-pencil environment, we 
conceive of instructional supports to include a sequence of 
tasks, techniques, and theory that inform the teacher’s 
instructional moves and the opportunities for students’ 
learning experiences. This research addresses the question: 
How is instruction in a CAS/paper-and-pencil environment 
supportive of students’ change in representational fluency in 
solving linear equations? 


Conceptual Framework 

We employed a learning progressions approach to 
elaborate both means of support for student learning and 
the nature of students’ learning vis-a-vis change in 
students’ representational fluency. Shown in Figure 1, a 
curricular framework informs the curricular progression 
and assessment of learning; the instructional sequence is 
informed by the curricular progression. Likewise, 
characterizations of student thinking are conceived of with 
respect to the influence of the instructional sequence and 
the manner in which learning is assessed. Our learning 
progression approach builds from an émergent perspective 
(Cobb & Yackel, 1996) in which means of support for 
students’ learning (i.e., curriculum, instruction including 
classroom interaction and discourse) inform and are 
informed by individual learning processes. In this article, 
we seek to provide support for the ways in which the 
instructional sequence influenced students’ representational 
fluency. This focus is shown as a dashed arrow in Figure 1. 
Means of Support 

Following Fonger, Stephens, Blanton, Isler, and Knuth (in 
press), a curricular framework specifies the targets of learning 
and entails big ideas, algebraic thinking practices, core 
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Figure 1. A learning progressions approach to investigating relations 
between means of support and student learning. 


assumptions about learning, and learning goals. This research 
is rooted in two big ideas of algebra: (a) equivalence, 
expressions, equations, and (b) a functions approach with 
multiple representations. The learning goal that guided the 
instructional design relevant to this report is to develop 
representational fluency in solving problems involving linear 
equations in one and/or two variables. Our assumptions about 
learning were rooted in an emergent perspective in which 
interactions and discourse at the classroom level inform and 
are informed by students’ activity and understandings. 
Specifically: (a) a meaningful understanding of equation 
solving is possible when explored through multiple 
technology-based representations (e.g., Kieran & Sfard, 
1999), and (b) a relational understanding of the equal sign is 
significant for students’ abilities to solve equations (e.g., 
Knuth, Stephens, McNeil, & Alibali, 2006). 

The curricular progression includes a sequence of lesson 
objectives that address mathematical content, role of 
representations, and tool use. Support for teaching includes 
summaries of empirical research to inform the design and 
implementation of tasks with students. The instructional 
sequence is a string of tasks specifying “what”? mathematics 
students are to accomplish, the techniques or “how to” 
complete these tasks with CAS/paper-and-pencil, and the 
domain-specific theory or core concepts that undergird the 
“mathematical understandings.” 

Characterizing Sophistication in Representational 
Fluency 

A representation is something that stands for something 
else from some one’s point of view. In solving equations 
from a functions approach, students may employ strategies 
within and across symbolic, graphic, and numeric 
representation types (including CAS/paper-and-pencil 
function tables and numeric equations such as 0=0). 


31 


Table 1 
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A Taxonomy for Sophistication in Representational Fluency 


Sophistication 


Lesser 


Greater 





Nature of Students’ 
Discursive Activity 


Representation Types 
One 


Multiple 


Student expresses an incorrect or incomplete 
creation, interpretation, or meaning. 


Student expresses correct and complete 


creations, interpretations, and meanings. 


SOLO Level 


Prestructural. Student demonstrates some 


procedural work within a given representation 


type but no understanding is conveyed. 
Student is not able to create and/or interpret 
representations of a given type with success 
to complete a task. 

Multistructural. Stadent moves between more 
than one representation type to make progress 
toward or successfully complete a task but 
demonstrates no or incomplete understanding 
of how these representation types are related 


Unistructural. Student creates and/or interprets 


representations within a single type to 
successfully complete a task but demonstrates 
no or limited understanding of connections to 
other representation types. 


Relational. Student relates at least two different 


representation types together with a more 
sophisticated understanding of how they are 
related, what they mean, and/or of their 
significance to the whole. In connecting 


or what they mean to solve the problem. 


Additionally, a unique feature of CAS technology is the 
result of evaluating an equation at values; CAS feedback 
yields the word “true” when an equation is evaluated at a 
solution to the equation, and “false” otherwise. The use of 
such CAS feedback to inform student learning of 
equivalence has been productive in previous research, and 
identified as an area for further investigation (e.g., Kieran & 
Drijvers, 2006). With an intention to build on the 
aforementioned studies by Kieran and colleagues, and 
following Kaput (1992) on the importance of studying 
students’ representational practices with technology, in this 
study we treat the CAS feedback of “true” or “false” as an 
important representation type. 

We characterize sophistication in students’ representational 
fluency based on the number of representation types, the 
correctness of students’ creation and/or interpretation of these 
representations in solving problems, and the meaning 
students making of their strategies. The structure of observed 
learning outcome (SOLO) taxonomy (Biggs & Collis, 1982) 
was adapted for this study to categorize levels of 
sophistication in representational fluency. In this case, a 
single assessed component is students’ use of a representation 
type to solve a problem. The coding taxonomy is given in 
Table 1; greater sophistication is discerned from left to right. 

In brief, when a student’s strategy focuses on a single 
representation type to solve a task, with little or no 
understanding of connections to other representation types, it 
may be an incorrect creation and _ interpretation— 
prestructural level. A correct creation and interpretation 
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multiple representations, a student gives a 
correct interpretation of an invariant feature 
across multiple representations or types. 


within a representation type is unistructural. When a 
student’s strategy involves more than one representation 
type, it could yield progress yet not successfully solving the 
task—nultistructural level. Finally, a student’s correct 
solution of a task with multiple representations with a more 
sophisticated understanding of how they are related, what 
they mean, and/or of their significance to the whole is 
relational. Students make a relational connection when they 
give a correct interpretation of an invariant feature across 
multiple representations or types. A typical example of a 
relational connection with CAS/paper-and-pencil is to solve 
7x + 3 =2x—7 for x by creating a graph of f(x) = 7x + 3 
and g(x) = 2x — 7 and interpret the meaning of the x-value of 
the intersection (x = —2) as the solution to 7x + 3 = 2x — 7. 
Another example of a relational connection is to type 
“Tx + 3 = 2x —7\¢=—2” into CAS to yield “true” and 
interpret the meaning of this movement from a symbolic 
equation to CAS feedback as signifying the solution to the 
equation is x = —2. Overall, change in greater sophistication 
in representational fluency is interpreted as a shift in 
discursive activity from incorrect to correct creation, 
interpretation, and meaning making within and across 
representation types. The meaning a student draws from their 
creation and/or interpretation of external representations is 
key at both unistructural and relational levels. 


Methods 
To understand both how to support students’ 
representational fluency in a CAS/paper-and-pencil 
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Table 2 
An Overview of the Instructional Sequence 
Sequence Topic and Representation Types Tools* 
Section 1 Equivalence of Expressions 
Activity 1 Justify equivalent expressions as the same pattern, graph, table CAS/P&P 
Activity 2 Justify equivalent expressions by common symbolic form CAS/P&P 
Section 2 Transition from Expressions to Equations 
Activity 1 Explain symbolic identity equations as numerically “always true” CAS/P&P 
Activity 2 Classify equations as sometimes (intersect), always (same line), or never true (parallel) CAS/P&P 
Section 3 Solving Linear Equations 
Activity 1 Solve equations ax + b = c OR a(x + b) = c for x with one solution 
Part I Compare graphs/tables fx) = ax + b or f(x) = a(x + b) and g(x) =c CAS/P&P 
Part II Define and verify solutions as “true” values CAS 
Part III Create equivalent symbolic equations P&P 
Part IV Connect a solution across graphs, tables, symbols, words CAS/P&P 
Activity 2 Solve equations ax + b = cx + d or ax + bx + c= dx + e for x (one solution) 
Part I Create a series of symbolic equivalent equations CAS 
Part II Create a series of equivalent equations, verify the solution CAS/P&P 
Activity 3 Solve equations ax + b = cx + d for x with one solution 
Part I Compare graphs/tables of f(x) = ax + b and g(x) = cx + d to solve CAS 
Part II Create a series of equivalent symbolic equations P&P 
Part III Connect “true” value across graphs, tables, symbols CAS/P&P 
Activity 4 Solve equations ax + b = cx + d for x with infinite/no solutions 
Part I Compare graphs/tables of f(x) = ax + b and g(x) = cx + d to solve CAS 
Part II Create a series of equivalent symbolic equations P&P 
Part II Connect “true” value(s) (if any) across graphs, tables, symbols CAS/P&P 


*Tools include computer algebra systems (CASs) and/or paper-and-pencil (P&P). 


environment, and the nature of learning in the context in 
which it was supported, we conducted one cycle of a 
collaborative teaching experiment (CTE) (Cobb, 2000), a 
type of design experiment (Gravemeijer & Cobb, 2006). 
This methodological choice was rooted in our intent to 
advance an understanding of how instructional supports 
influenced students’ representational practices. Student 
interviews were conducted before and after the teaching 
experiment. 

Collaborative Teaching Experiment 

We conducted a five-week CTE in three phases: design; 
experimentation and ongoing analyses; and retrospective 
analyses. The CTE design was chosen to: (a) address a 
problem of practice relevant to classroom teaching; (b) 
enhance the ecological validity of the findings per the 
research setting of an intact classroom; and (c) provide 
potentially meaningful supports to bolster opportunities for 
students to learn with CAS. 

Participants. All student participants (n=31) were 
enrolled in the same ninth-grade algebra course in an urban 
public high school in the Midwest of the United States. 
Ms. R, a fifth-year high school mathematics teacher taught 
the instructional unit during the last period of the school 
day. Three students from this class volunteered to also 
participate in task-based interviews before and after the 
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instructional unit, and have their daily classwork recorded. 
We selected three students, Annie, Bryon, and Carlos,” who 
demonstrated a range of mathematical abilities (mid, high, 
and low percent scores, respectively) relative to a class- 
wide pre-assessment. 

Instructional sequence. The instructional design was 
guided by a consideration of how core concepts, 
techniques, and tasks encouraging using multiple 
representation types and tools could be coordinated and 
sequenced to support student learning. Table 2 outlines the 
content, representation types, and tools used during the 
instructional sequence. The focus of this study, Section 3 
Solving Linear Equations, was introduced by building on 
students’ experiences in comparing and _ justifying 
expressions related by an equal sign to identifying solutions 
to equations with one solution such as ax+b=c and 
aXx+b)=c, or axt+b=cx+d with functions f(x) = 
ax+b or fix)=ax+b) and g(x)=c or g(x)=cx+d, 
respectively. Tasks involved creating and comparing CAS 
graphs and tables of equations as comparisons of functions 
before symbolic manipulations with CASs and paper-and- 
pencil; the CASs with operator “|” was used to verify 
value(s) that make an equation “true” (i.e., solutions to 
equations). Symbolic transpositions with step-by-step 
equation solving techniques of applying the distributive 
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Table 3 
Classifying Solution Sets to Linear Equations 


a 


Sample Tasks 


Answer the questions for each 
equation (A, B, and C): 

1. The equation is sometimes, 
always, or never true. 

The equation has____ 
solutions. I know this 
because __. 

A. Graph of 
4x + 1 =2(2x + 0.5) 

B. Table of x + 2 = 2x - 
x+4 

Gras = —x +3 + x 


overlapping 


and graphs. 


and Exp2. 


Techniques 


e Translate from a symbolic equation in x: 
ax + b = cx + d to graphs and tables of 
fx) = ax + band g(x) =ext+d 

e Interpret graphs as intersecting, parallel, or 


e Classify solution sets as one, infinite, or zero 
by interpreting equations in symbols, tables, 


Theory (Concept) 


1. Solutions to equations can be 
determined by equality of 
expressions. Linear equations are 
relations between linear expressions 
that are sometimes, always, or never 
equal in value. Thus linear equations 
have one, infinitely many, or zero 
solutions, respectively. 


e Interpret an equation Expl = Exp2 asa 
relationship between two expressions Expl], 





property, combining like terms, and isolating a variable 
were introduced on CASs, then paper-and-pencil. Finally, 
instruction on solving equations with infinite and no 
solutions followed a similar instructional pattern— 
translating from symbolic equations to Cartesian graphs 
and function tables with CASs before symbolic 
transpositions with paper-and-pencil or CASs. The tasks, 
techniques, and conceptual learning goals relevant to 
solving equations are explained next. 

Introduced in Table 3, the central idea of Concept 1 is the 
notion that students must come to understand the equal sign 
as an equivalence relation in to make sense of solving linear 
equations in one variable (cf. Knuth et al., 2006). When an 
equation in one variable is viewed as an equivalence relation 
it means that the equation is a statement of a relationship 
between two expressions, Expression, = Expressionz, that is 
sometimes, always, or never true; expressions can be 
compared as functions in which equivalent expressions 
signify the same function in symbols, graphs, tables, and 
words (Kieran & Sfard, 1999; Yerushalmy & Gilead, 1997). 
In the context of solving linear equations, this means an 
equation has one, infinite, or no solutions, respectively. This 
view was emphasized by Kieran and Drijvers (2006) in their 
teaching experiment with tenth-graders in which “the 
relation between two expressions being equivalent or not, 
and the corresponding equation having many, some, or no 
solutions was explored in both CAS and paper-and-pencil 
tasks” (p. 216). For example, tasks included classifying the 
truth of solutions, and justifying the number of solutions by 
examining the equivalence or nonequivalence of expressions 
related by an equal sign using graphs, tables, and symbols. 
The central design principle was to support students in 
linking equivalence of expressions to understanding 
equations in one variable as equivalence relations before 
solving equations. 
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Two additional concepts are introduced in Table 4. 
Concept 2 is based on the notion that in solving equations 
from a functions approach, students needed to understand 
connections among representations to coordinate the 
meaning of a solution across representation types (Huntley 
& Davis, 2008; Huntley, Marcus, Kahan, & Miller, 2007). 
Tasks were designed to support students’ techniques of 
connecting multiple representations, especially in justifying 
a solution across symbols, graphs, numeric tables, and CAS 
feedback of “true.” For example, a task was to solve an 
equation in multiple representations then articulate how the 
solutions were related. 

To support the coordination of information across 
multiple tools including paper-and-pencil and CAS-based 
representations, students should interpret, reflect on, and 
make sense of differences among those representations 
(e.g., Dick & Edwards, 2008). Instruction was designed to 
support learning experiences that regularly involved the use 
of both CASs and paper-and-pencil tools. Emphasis was 
placed on the techniques of translating, transposing and 
interpreting across and within multiple representations, and 
reconciling across tools. For example, with Concept 3, 
students translated from symbolic equations to graphs and 
tables to predict solutions to equations with no or infinite 
solutions before establishing the solution set. 

Experimentation and ongoing analyses. Authors N. 
Fonger and M. Rohwer (hereafter referred to as Ms. R) co- 
planned all lessons. Ms. R taught all lessons, which followed 
a common pattern: introduction/warm-up problem, check 
homework, in class activity, summary/exit ticket. N. Fonger 
led the development of curriculum and instructional design, 
ongoing and retrospective analyses, and was a participant 
observer in the classroom, also managing all data collection. 
After each teaching episode, N. Fonger and Ms. R met for 
one to two hours to conduct ongoing analyses of the 
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Table 4 
Solving Linear Equations Using Multiple Representations 


Sample Tasks 


Solve 4x — 9 = —7x + 13 for 


x in graphs, tables, and 
symbols. How does the 
solution found in graphs 
relate to the solution found 
in tables? In symbols? 
Check using CAS | 
operator. 


For each Equation (D and E): 


F 


Use a graph or table to 
predict if the equation has 
zero, no, or infinite 
solutions. Provide a sketch 
to show why. 


Supports for Representational Fluency 


Techniques 


e Solving by symbolic transpositions (e.g., for 
4x -—9 = —7x + 13, add 9 to obtain 
4x = —7x + 22, add 7x to obtain 11x = 22, 
divide by 2 to obtain x = 2). 

e Solving by translating to a graph (e.g., create 
a Cartesian graph of f;(x) = 4x — 9 and 
Fox) = —7x + 13, (2, —1) is the point of 
intersection, interpret x = 2 is the solution) 

e Solving by translating to a table (e.g., create 
function table for f(x) = 4x — 9 and 
Sox) = —7x + 13, when x = 2, 

Si(x) = —1 = f(x), interpret x = 2 is the 
solution). 

e Solving by translating to CAS verbal output 
(e.g., evaluate equivalence on CAS: 4x — 

9 = —7x + 13|x = 2 gives output “true”, 
interpret x = 2 is a solution) 

e Connect solutions across representation types 
(e.g., x = 2 was “true” on CAS and also the 
value that makes /;(x) = f2(x) on the graph 
and table). 

e /nterpret symbolic equations, and graphs and 
tables of functions to represent infinite, one, 
and no solutions to an equation (e.g., 
6x = 5x + 7 + x has no solutions because the 
graphs f)(x) = 6x and A(x) = 5x + 7+ x are 
parallel, and the tables never have the same 


2. Solve the equation for the value). 

variable. Show your work. e Transpose within symbolic representations on 
3. Check your solution. Does paper-and-pencil and CAS to create a series 

it make sense? of equivalent symbolic equations (e.g., 
4. Was your prediction 6x = 5x + 7 +x can be simplified to 


correct? Explain why or 
why not. 


Da6c— oxic 
E. -s—4=-—l(s+4) 


6x = 6x + 7, subtract 6x to get 0 = 7, this 
equation is never true, thus there are no 
solutions) 

e /nterpret the solution set as the value(s) that 
make the equation true (if any), Justify the 
solution using graphs, tables, symbols (e.g., 
—s—4=-—1(s + 4) when graphed as 
Kis) = —s — 4 and g(s) = —1(s + 4) is the 
same graph, so —s — 4 = —1(s + 4) has infinite 
solutions. Also —s — 4 is equivalent to 
—1(s + 4) so the equation —s —4 = —1(s + 4) 
is always true). 


Theory (Concepts) 


2. Different representations can signify 
the same object. Different representations/ 
representation types of the same linear 
expressions, equations, signify the same 
relationship, pattern, function, or solution 
set from different yet complementary 
perspectives. 

3. Solving equations in one variable is 
conceptualized as a comparison of two 
functions. Linear equations in one 
variable such as ax + b = cx + d for real 
valued parameters a, b, c, and d, can be 
solved for the variable x by comparing the 
functions f(x) = ax + b and g(x) =cx +d 
for the value of x that makes the equation 
ax + b= cx + d true. Graphical, tabular, 
or symbolic methods can be used. 


instructional episode using lesson artifacts, and student 
formative assessment items as data sources. The focus of 
these meetings was to conjecture and substantiate 
productive means of support for students’ development of 
representational fluency; N. Fonger summarized these 
analyses and shared these with Ms. R as a member- 
checking procedure to improve cycles of design, and 
future experimentation and analyses. Consistent with 
CTE method, teaching activities were developed and 
evolved throughout the experiment to support the 
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evolving understandings of the participants (Cobb, 2000). 
Our daily analysis of teaching episodes and conjectured 
instructional supports led to some changes in instructional 
design, especially in supporting students’ interpretation of 
CAS-based representations. For example, Ms. R reflected 
that students seemed confused at how to interpret CAS 
function tables that displayed two functions at once, so 
we created an instructional aid displaying two tables— 
each with x and f(x) value pairs—to solve an equation 
with tables. 
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Solve for the variable and check your solution using a table, graph, or equation (your choice). Sketch the 
section of the graph (with the scale marked) or the table and indicate where you found the answer. If you 
solve the problem using equations, show all of your work. 

t-2+3t=-6+4t+4 fort 


Figure 2. An initial interview task. 


Retrospective analyses. During the third phase of the 
design experiment, N. Fonger conducted retrospective 
analyses of teaching experiment data. This involved 
analyses of classroom video and triangulation with analyses 
of additional data: student work, classroom observation 
field notes, and reflections from daily meetings between the 
teacher (Ms. R) and researcher (N. Fonger). The analysis 
process during this stage followed a constant comparative 
method (Glaser & Strauss, 1967); with each day taken in 
succession, conjectures related to a path of reasoning in 
classroom instruction were confirmed or refuted based on 
analyses of the aforementioned data sources, which were 
then tested against the subsequent day’s data. 

Video data were segmented into critical moments— 
smaller episodes of instruction that supported or refuted the 
conjecture instructional theory. These critical moments 
were coded according to the core concepts and techniques 
that structured classroom activity in support of 
representational fluency (as introduced in Tables 3 and 4). 
Change in Representational Fluency 

Each of Annie, Bryon, and Carlos participated in an 
individual 50-minute semi-structured task-based interview 
with N. Fonger at the beginning and at the end of the 
teaching experiment. Students solved a series of problems 
involving linear equations with various solution types (i.e., 
one, none, or infinite solutions). Solving a symbolic 
equation with infinite solutions is a task type previously 
identified to pose great challenges for students (e.g., 
Huntley et al., 2007). The initial and final interview tasks of 
focus are given in Figures 2 and 3. By design, students had 
considered similar tasks during the respective pretests and 


Steps 


posttests. The focus of the interviews was to discern 
students’ representational fluency in solving these 
equations. Students were encouraged to “think out loud” as 
they created, interpreted, and connected representations. 

Analyses of video of semi-structured interviews were 
triangulated with analyses of written and dynamic 
inscriptions students created with paper-and-pencil and 
CAS during the interviews. Analyses of each student’s 
strategies were coded per the framework for sophistication 
in representational fluency (Table 1) and were analyzed in a 
cyclical fashion across tasks, and interviews. Due to the 
scope and timeline of this study it was not feasible to track 
changes in students’ levels of sophistication in 
representational fluency across each day of the teaching 
experiment. However, careful analyses of Annie’s, 
Bryon’s, and Carlos’s CASs and_ paper-and-pencil 
inscriptions on classroom activities were conducted both (a) 
during ongoing analyses of teaching experiment data as 
reflected in daily summaries to document the impact of the 
curricular and instructional supports on students’ change in 
representational fluency, and (b) during retrospective 
analyses to garner supporting or refuting evidence of means 
of support. 


Results 
This section elaborates the nature of instructional 
supports for students’ change in representational fluency in 
solving linear equations with CAS/paper-and-pencil. 
Instructional Support 
Three key means of support are reported: (a) translation 
from symbolic equations to graphs and tables was 
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1.) Original problem 


2-x-x=x+8-3x-6 





2.) Combine like terms 


2-2x=-2x+2 





3.) Add 2x to both sides 


2=2 





4.) Subtract 2 from both sides 


0=0 





5.) Solution 








x=0 andx=2 











Andy worked out the problem above. When he substituted his solutions into the original equation, he 
discovered that both x = 0 and x = 2 make the equation 2—x-x =x+8-—3x—6 true, 

a) Was Andy’s work correct? Explain why or why not. 

b) What is the solution to the equation 2—x-x=x+8-3x-69 








Figure 3. A final interview task. 
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3. Use Markus' method to find a solution to 8 = 2x - 6. Show your work 
below. 


{ 





Figure 4, Annie’s use of graphs, tables, and symbols to solve a linear 
equation. 


sequenced before symbolic transpositions; (b) solutions 
were interpreted as invariant features across representation 
types; and (c) expressions relating an equal sign were 
compared from multiple representations. Evidence is drawn 
from analysis of the third section of the instructional 
sequence focused on equation solving (Table 2, Section 3, 
Activities 1-4). 

Translations from symbols to graphs and tables 
prioritized. In the teaching experiment, we capitalized on 
CAS as a representational toolkit to create and interpret 
graphs and tables. When a functions approach to solving 
was introduced, instruction emphasized first a translation 
from a symbolic equation to a Cartesian graph, then a 
function table before transposing within the symbolic 
representation type. Annie’s equation solving strategies in 
Figure 4 reflects the typical instructional approach; from 
left to right, Annie first translated from a symbolic equation 
to a graph and a function table, then created equivalent 
symbolic equations. 

The approach to translate from a symbolic equation to a 
graph comparing two functions and corresponding table 
before formal symbolic manipulations on equations was a 
continued theme when solving more complicated equations 
with variables on both sides of the equal sign (of the form 
ax + b=cx +d with one solution). For example, consider 
Vignette 1. 


*Unsaved <> 
x TOs V 20K = 
A%x-9 > 3 





Vignette 1. The task is to “Solve the equation for the variable. Show 
your work. Check your solution. 4x — 9 = —7x + 13” A typical approach 
to solving equations was to translate to graphs and tables before 
performing symbolic transpositions. 

Ms. R: “So, if I, I’m not sure by looking at these equations (in 
reference to 4x — 9= —7x + 13), I can graph these in my CAS, What 
happened to those lines?” 

Katrina: “They overlap.” 

Ms. R: “They cross don’t they? They sure are intersecting. Alright. So 
these guys are crossing. They have a point of intersection. They have one 
solution. And if I look at my graph can I figure out what value of x 
they’re going to cross at?” 

Abila: “Negative one and two.” 

Ms. R: “So x = 2, y= —1. What if I look at the table?” 

Davon: “You can check.” 

Ms. R: “That’s the way we do it, too. So if I go to my table, Control T, 
go to your table, and look where my y-values are the same, OK, that’s the 
only place where the y-values are the same. Right here (gestures to table) 
x = 2, y= —1. That’s the only place. So one solution.” (CTE, 10/11/11) 





Annie’s translation from a symbolic to graphic and 
numeric representations for this task is given in Figure Sa). 
Annie’s paper and pencil work shows that she identified 
“2” as a “sometimes” solution to 4x — 9 = —7x+ 13 (in 
Figure 5b). We interpret this as a reflection of the 
instructional sequence and language used when identifying 
an equation with one solution as being “sometimes true.” 

Instruction on how to solve equations with graphs and 
tables of two functions occurred first. This was followed by 
instruction on solving equations in a step-by-step symbolic 
fashion (motivated by a student sharing his work, see 
Section 3, Activity 1). Next, instruction targeted the 
coordination of CAS/paper-and-pencil to create a series of 
equivalent symbolic equations of the form ax + b=cx+d 
(see Section 3, Activity 2). Finally, the use of CAS 
techniques to solve equations in a step-by-step fashion to 
create a series of equivalent equations was introduced. 

Solutions are interpreted as invariant features across 
representations. Connections among representation types 
were explicitly emphasized when solving equations. For 
example, we saw evidence of this instructional emphasis in 





Z x Some 4 the ® 


Figure 5. Annie’s approach to solving a linear equation with (a) CAS and (b) paper-and-pencil. 
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the aforementioned work of Annie in Figures 4 and 5. 
Moreover, in what follows, Vignette 2 illustrates how 
instructional supports also emphasized leveraging CAS 
feedback to discern the truth of an equation at a value: 


Vignette 2. After identifying the x-value of 26 from the graph and table 
of f4(x) = 10 — 3x and f5(x) = 22, Ms. R introduced a CAS technique of 


evaluating a symbolic equation at a numeric value to yield a verbal 
representation of “true,” interpreting x = 26 as the solution to the equation 
22 = 100 — 3x (see Figure 6) (CTE, 10/5/11). 





Graphic and numeric approaches were also coordinated 
with solutions determined by symbolic transposition. For 
example, in reference to the aforementioned functions 
approach to solving 4x — 9 = —7x + 13 as in Vignette 1 
and Figure 5, Ms. R articulated: 


Now let’s look at our other representations (slides CAS 
screen as in Figure 5a next to symbolic transpositions, 
not shown) At x = 2, (points to x-value of 2 in table, 
Figure 5a) that was my solution here (points to values of 
fi(2) and f,(2) in table), where these lines crossed 
(points to (2, —1) in graph, Figure 5a), and we did it 
symbolically. We did it all three ways. So we got the 
graph, we got the table and we can see from the rule, 
that’s the same thing as our equation. (CTE, 10/11/11) 


From these examples we see that classroom and 
individual activity promoted an explicit focus on creating 
and interpreting the solution as an invariant feature of a 
symbolic equation or pair of functions across graphs, tables, 
symbols, and CAS feedback of “true.” 

Comparing expressions related by an _ equal 
sign. The approach to comparing expressions related by an 
equal sign using graphs and tables was followed for 
equations with infinite and no solutions as well. In Section 3 
Activity 3, language of “‘one solution, infinite solutions, or 
no solutions” was used to parallel and build on the approach 
to comparing expressions related by an equal sign in Section 
3 Activity 2 with the language of “sometimes, always, or 
never true.” Students were often given choices for which 
representation type to use to justify their solutions. For 
example, Annie’s work often demonstrated her preference 
to describe the nature of a solution set by interpreting 
graphic representations. For example, she crossed her hands 





22=100-3-x[x=26 


Figure 6. A solution to an equation evaluates as “true” on CAS. 
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like an “X” explaining that a solution was “where both of 
them, kind of like, intersect... this is that sometimes thing.” 
Other students, such as Bryon, often chose to perform 
symbolic transpositions. Finally, in Vignette 3, instruction 
targets symbolic equivalence of expressions. 


Vignette 3. Ms. R: “Infinite solutions. Why? Why is it infinite 
solutions? Cause it means the same thing on either side of the equal sign. 
Look back at this step right here (-~r — 5 = —r—5). What do you see 
about the expression on the left and the expression on the right?” 


Ethan:“They’re the same.” 

Ms. R: “They’re exactly the same. So you have equivalent expressions. 
If your expressions on either side of the equals sign are exactly the same 
that means you have the same line, you have exactly the same line, so 
infinite solutions.” (CTE, 10/13/11) 





In the above exchange the teacher focused students’ 
attention on interpreting equivalent expressions related by 
an equal sign as being the same across symbolic and graphic 
(i.e., “same line’’) representation types. Also, the equation 
was Classified as having “infinite solutions.” We note that 
the instructional strategies in Vignettes 1 and 3 provided few 


Table 5 
Students’ Change in Representational Fluency. Solving a Linear Equation 
with Infinite Solutions 





Sophistication 
Lesser Greater 
Nature of Student expresses Student expresses 
Students’ an incorrect or correct and 
Discursive incomplete creation, complete 
Activity interpretation, creations, 
or meaning. interpretations, 
and meanings. 
Interview SOLO Level 
Prestructural Unistructural 
Initial ABC 
AB 
B 
B 
Final ABC A 
GC 
B 
Multistructural Relational 
Initial AB 
AB 
A 
Final B 
B A 
B AB 
B 


$e 
A single letter is used to denote a strategy for each student 
(A= Annie, B = Bryon, C = Carlos). Multiple letters indicate 
multiple strategies for a given student. 
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Opportunities for students to explain their thinking and to 
reflect on and justify their work; an issue we discuss later. 
Students’ Change in Representational Fluency 

In light of the results of the instructional context, we now 
turn to an account of students’ change in sophistication of 
representational fluency. Table 5 summarizes results of 
students’ sophistication in representational fluency in 
solving the tasks given in Figures 2 and 3. In the initial 
interview Annie and Bryon both demonstrated prestructural 
and multistructural levels of representational fluency, while 
by the final interview Annie demonstrated unistructural, 
and both Annie and Bryon demonstrated relational levels. 
Carlos persisted at the prestructural level. 

At the onset of the teaching experiment, all three students 
were characterized to have low levels of sophistication in 
representational fluency in solving equations with incorrect 
creations and/or interpretations of representations. Consider 
Annie’s work in Vignette 4. 


Vignette 4. Annie attempted to solve t- 2+ 3t= —6+ 4+ 4 for t. 
Annie was initially stuck within the symbolic representation type, then 
N. Fonger asked Annie if she could compare t— 2 + 3t and —6+ 4t+4 
as a graph (of functions of #). Annie created graphs using her graphing 
calculator and paper-and-pencil (shown in Figure 7a, b). 

Annie reflected, “How they (Figure 7a, b) equal the same thing I have 
no idea.” 

N. Fonger prompted, “So if you were to write down what’s the solution 
to that equation, what would you write down?” Annie replied, “I have no 
idea what the solution would be. The solution’s like, it’s the same thing, 
it’s the same thing, it should probably equal the same thing in graph and 
in table, but right here (points to her symbolic work that shows 7¢ + —4), 
it doesn’t actually equal to the same thing.” 

Annie’s final strategy was to work within the symbolic representation 
type, making a computational mistake in adding like terms (see Figure 
7c). This work led Annie to conclude that t= —4 was the solution to the 
equation (Initial Interview, 9/13/11). 





Like Annie, Bryon and Carlos demonstrated inconsistent 
and incorrect strategies for combining like terms in 
symbolic equations. For Bryon, he had initially been 
successful in combining like terms with paper and pencil to 
reach the equation “3T=3T.” From this equation he 
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concluded, “Oh, I think I messed up somewhere in there.” 
Thus despite a correct creation of an equivalent equation, 
he was not able to draw meaning about the solution to this 
equation, a prestructural level of fluency. Bryon followed 
this with an additional strategy in which he incorrectly 
combined like terms to reach 0 = 2T, concluding, “I figured 
that all the ¢’s are zero.” Carlos used his graphing calculator 
to add the numbers in t — 2+ 3t=—6+4t+2 because 
“that’s what my eighth-grade teacher told me—you don’t 
have to put a bunch of variables and letters in there; you 
just add the numbers.” He concluded the equation was 
“false”; one equation can’t equal another. 

With respect to multiple representations, both Annie and 
Bryon were able to graph two lines on their calculator when 
prompted, yet the “sameness” they observed was not yet 
meaningful with respect to the solution to the equation. The 
ability to correctly create representations, moving across 
representation types, without a correct interpretation of the 
meaning is a multistructural level of representational fluency. 

In the final interview, Annie was initially unable to make 
sense of Andy’s work within the symbolic representation 
type (a fictional student, as given in the problem statement), 
yet leveraged graphs and CAS feedback to make progress 
as seen in Vignette 5. 


Vignette 5. Without explicit prompting, Annie decided, “If I take both 
of them (points to 2 — 2x and —2x + 2) which is probably going to be 
equal in the graph, it may help (creates graph, Figure 8a). They have 
(creates table, Figure 8b) infinite solutions (cursors in table from fo(0) = 2 
to f7(0) = 2).” 

Annie later decided to insert a CAS calculator page in which she 
evaluated the original equation at the values of x= 0 and x= 2 [Figure 
9]). She claimed “They’re both true... each one is correct.” 

Prompted to explain to someone else what the solution was, Annie 


replied, “It could be either zero, or two.” 

With further probing if the solution “could it be anything else,” Annie 
replied, “Yes, including that it’s a very infinite solution I believe. 
(Switches back to split graph and table view on CAS like Figure 8, and 
scrolls down to f7(3)) ... Oh I get what they’re doing (points to 
2 —2x = —2x + 2 and 2 = 2), they’re adding 2x to both sides ... Oh I get 
it, because it’s infinite solution. They’re both the same, except for with 


minuses. ... Yeah, they’re infinite solutions it’s hard to tell what x 
equals.” (Final Interview, 10/18/11). 
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Figure 7. Annie’s (a) graphing calculator graph, (b) paper-and-pencil graphs, and (c) symbolic equation solving activity during the initial interview. 
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Figure 8. Annie’s (a) graphic and (b) split screen graphic/numeric equation solving activity during the final interview. 


Annie was successful in drawing meaning about the 
solution set to the equation from graphs and tables, and 
coordinating the meaning of x=0 and x=2 across the 
symbolic equation and CAS feedback of “true.” When she 
returned to the symbolic equation, she recognized the 
“sameness” in the expressions related by an equal sign, 
which meant infinite solutions. One way to interpret 
Annie’s last statement, “It’s hard to tell what x equals,” is 
the meaning of “infinite solutions” is not well connected to 
making statements about the solution set, which is 
commonly expressed in the form of a statement that 
“x=__” (a number). Annie’s meaning for “infinite” 
solutions was conveyed when interpreting graphical, 
numerical, and symbolic representation types, in addition to 
the CAS feedback of “true,” yet conventional language 
about the solution set as being all real numbers was not 
evident, a point of confusion for Annie. Like Annie, Bryon 
demonstrated change in representational fluency at the 
relational connection level by using CAS feedback to 
confirm x = 2 is true, concluding, “It is a solution, but I 
don’t know how.” After creating CAS graphs of 
fax) =2—-—x—-—-x and f4(x)=x+8 —3x—6 he noted the 
graphs “overlapping the whole time” yet insisted the 
solution was “zero equals zero [0=0]” without 
recognizing infinite solutions (a multistructural level). 

In contrast to Annie and Bryon, Carlos did not make 
progress beyond the prestructural level. In reading the 
problem statement, Carlos did not think it was possible to 
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Figure 9. Annie’s symbolic to verbal translations on CAS to confirm 
solutions to a linear equation during the final interview. 


40 


have two solutions. He expressed confusion about the 
problem statement, “Since the x equals zero and x equals 
two, because if x equals zero how can x equal two?” 
Prompted to use his CAS, he typed “0” Enter, then “2” 
Enter. He did not interpret these as solutions to equations. 


Discussion 

Supporting students’ representational fluency and 
meaningful equation solving in technology-rich settings is 
an important and timely issue (Arbaugh et al., 2010; NRC, 
2001). In this exploratory study we employed a learning 
progressions approach to elaborate both means of support 
for student learning and the nature of students’ 
representational fluency. We found instructional supports to 
be productive for some students: (a) translation from 
symbolic to graphic and/or numeric representation types 
was prioritized before symbolic transpositions in solving 
equations, (b) connecting solutions across CAS and paper- 
and-pencil inscriptions including CAS feedback of “true,” 
and (c) the treatment of the equal sign as a relational 
symbol in comparing expressions that are sometimes, 
always, or never true. Evident throughout the CTE, and as 
measured in students’ equation solving activity in 
interviews, both Annie and Bryon advanced to more 
sophisticated levels of representational fluency in solving a 
linear equation with infinite solutions both within and 
across representation types, while Carlos’s representational 
fluency did not change. This finding has implications for 
research on how students’ sophistication in representational 
fluency is characterized. In-depth examinations of students’ 
problem solving may reveal nuances between levels of 
sophistication that shed light on how _ students’ 
understandings of mathematical ideas are intertwined with 
their representational fluency (e.g., see Fonger, Dogan, & 
Ellis, 2016). While levels above relational were not 
observed in this study, prior research has shown that older 
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students demonstrate sophisticated representational fluency 
in solving equations with CAS (e.g., Fonger, 2011). This 
stresses the idea that developing sophisticated fluency 
occurs over multiple years, not a single teaching unit. 
Instructional Supports for Representational Fluency 

Across these data we saw how instructional support and 
student activity can embody the core concept of an equation 
as a relationship between two expressions across graphs, 
tables, and symbols. For students such as Annie, we 
conjecture the comparison of expressions via multiple 
representation types throughout the teaching experiment 
was developmentally important, especially how the 
language of “sometimes, always, never true” was linked to 
language of “one, infinite, or no solutions.” Second, the 
instructional approach of comparing graphs and tables 
before symbols seemed significant in Annie’s choice to use 
graphs to identify solutions/solution sets to equations, yet 
not all students took up this strategy, nor in a meaningful 
way. For example, Carlos did not use multiple 
representations in the final interview, and while Bryon was 
able to create graphs from a one-variable equation he did 
not interpret the meaning with respect to solving the 
equation. Thus for some students, this study corroborates 
existing research findings suggesting the potentially 
meaningful sequence of graphic and numeric approaches to 
solving equations prior to symbolic transpositions (e.g., 
Kieran & Sfard, 1999; Yerushalmy, 2006; Yerushalmy & 
Gilead, 1997). Yet for other students, this instructional 
approach was not supportive. Third, the instructional design 
emphasized the use of the CAS feedback of “true,” which 
was highlighted in the final interview as a key way that 
Annie and Bryon demonstrated connections across 
representation types. The role of coordinating the CAS 
feedback of “true” or “false” with graphic and numeric 
approaches to solving equations resonated with students. 
However, for equations with infinite solutions the truth of 
one or two values needs to be situated with respect to an 
infinite solution set, a difficulty of solving this type of 
equation. 

The use of natural language to describe equivalence of 
“sometimes, always, or never true” in graphs seemed 
meaningful to students throughout instruction and in the 
final interviews. The use of CAS feedback of “true” for 
evaluating solutions seemed particularly supportive of 
students’ problem solving. This finding underscores the 
importance of addressing students’ representational fluency 
with natural language and technology feedback in the 
complex domain of mathematical equivalence and equation 
solving from a functions approach. 
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Improving Future Instructional Design 

To build upon this exploratory study, several cycles of 
experimentation are needed to refine instructional designs, 
and gamer empirical support for effective instruction linked 
to students’ development of representational fluency. In 
light of the finding that Carlos did not change his 
representational fluency, it is feasible to infer that the 
approach to sequencing and connecting multiple tool-based 
representations to understand equations as equivalence 
relations was not an effective instructional approach for 
him. In light of research on effective classroom interaction 
patterns (cf. Wood, 1998), we hypothesize that students in 
this study may have achieved greater sophistication in 
representational fluency had they been provided more 
consistent opportunities to explain their reasoning and 
discuss their own meaningful connections. Additionally, 
there is a need to better understand the nature of 
instructional supports for diverse populations of learners 
including how to cultivate effective student-tool relations 
(e.g., Kieran & Drijvers, 2006). How might CAS be further 
leveraged as a “partner” in students’ learning? 

Finally, we wonder how other noncognitive dimensions 
of the classroom environment might better support students’ 
learning. For instance, Carlos, like some other students in 
the class, had a relatively disengaged disposition during his 
day-to-day activity during the CTE, and seldom completed 
the in-class activities. Future instructional designs should 
attend to students’ motivation to solve problems with 
multiple representations. For example, how might a story, 
metaphor, or embodied approach support students to engage 
in meaning-making? How might task design and instruction 
support the “need” for bolstering representational fluency? 
In close, this research extends studies that focus on students’ 
difficulties in translating between representations to solve 
problems alone. Examining links between individual 
activity and viable instructional supports in complex 
learning environments is an important issue that warrants 
further investigation in research and practice. 
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The Going Green! Middle Schoolers Out to Save the World project aims to direct middle school students’ enthusiasm 
for hands-on activities toward interest in science and other STEM areas while guiding them to solve real-world problems. 
Students in this project are taught by their teachers to use energy monitoring equipment to audit standby power consumed 
by electronic devices in their homes and communities. Major findings were: (a) Beliefs in climate change increased more 
for students in the treatment than comparison group, pre to post; and (b) For girls there was a larger positive impact on 
climate change beliefs than for boys. These and additional findings presented in this paper provide evidence that a hands- 
on engaged-learning curriculum can have a positive influence on climate change beliefs and intentions and strengthen the 


association between the two constructs. 


While a majority (63%) of American adults report they 
believe in the existence of climate change, few (14%) say 
they are “very worried” about it (Leiserowitz, Maibach, 
Roser-Renouf, Feinberg, & Howe, 2013). Those surveyed 
see climate change as a distant threat that will impact future 
generations (Leiserowitz et al., 2013). Americans have been 
shown to be more skeptical than people in other countries 
(Carlsson et al., 2010; Infographic, 2011). A European 
study of 27 member states of the European Union (EU) 
revealed an overall belief that climate change is a serious 
issue (Eurobarometer, 2009). Researchers have pointed out 
a lack of curriculum content regarding climate change in 
schools (Choi, Niyogi, Shepardson, & Charusombat, 2010), 
leading to students receiving more of their information 
from the media than teachers (Robertson & Barbosa, 2015). 
The age-stability theory assumes that adolescents’ social 
and political attitudes are already strongly developed by the 
time they leave secondary school (Alwin & Krosnick, 
1991; Sears & Funk, 1999). Therefore, the development of 
positive environmental attitudes in childhood is an 
important element in shaping behaviors in later life 
(Ballantyne, Connell, & Fien, 2006; Chawla, 1999; 
Meinhold & Malkus, 2005). Researchers found a positive 
relationship between learning about the environment at 
school and the level of environmental concern (Barth & 
Michelsen, 2013; Hungerford & Volk, 1990; Knafo & 
Galansky, 2008). Researchers have concluded _ that 
educating students about. the environment can influence 
their future behaviors. Few studies have attempted to 
measure beliefs and intentions regarding climate change at 
the middle school level. 
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Literature Review 
Theoretical Background 

Intentions and beliefs have been recognized as important in 
science education for decades (Haney, Czerniak, & Lumpe, 
1996). In the theory of planned behavior (Ajzen, 1985), 
beliefs are used to predict an individual’s intention to engage 
in behavior. Behavioral intentions have been defined as the 
indication of an individual’s readiness to perform a given 
behavior, based on attitudinal beliefs and perceived behavioral 
control (Ajzen, 2002), and are assumed to be an immediate 
predecessor to behavior. The theory of planned behavior was 
developed from the expectancy value models (Ajzen & 
Fishbein, 1980) where if people evaluate a suggested behavior 
as positive (attitude) there is a higher intention (motivation) to 
perform the behavior. Many studies have confirmed a high 
correlation between attitudes and behavioral intention and 
subsequently with behavior (Sheppard, Hartwick, & 
Warshaw, 1988). One study measuring attitudes toward 
human-induced climate change found students who have a 
more accepting attitude toward climate change are more likely 
to express a willingness to take action (Sinatra, Kardash, 
Taasoobshirazi, & Lombardi, 2012), 

Fortus (2014) pointed out the lack of published research 
during the first decade of the 21st century related to the 
affective domain in science education. He further 
contended that very little learning occurs without a desire to 
learn (motivation, attitudes, beliefs, self-efficacy, interest, 
self-concept, and others): 


Regardless of whether students will go on to work in a 
STEM-related profession or just live in a STEM- 
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influenced world, we should strive for all to have 
positive attitudes to science and its role in society, 
motivation to understand the science of issues directly 
related to their lives and their general well-being, and a 
belief in their ability to make sense of issues. (p. 822) 


Measuring students’ attitudes toward science-related 
educational endeavors allows educators to plan appropriate 
instruction and determine effectiveness of the curriculum. 
Climate Change Engagement 

A comprehensive, international review of climate 
change engagement for young people revealed many 
studies that captured youth views on climate change but 
little research on ways in which this population could be 
more effectively engaged (Corner et al., 2015). Scholars 
agree that how individuals understand climate change is 
important in their willingness to take action (Lorenzoni 
& Pidgeon, 2006). However, much research has also 
shown that knowledge about climate change does not 
necessarily lead to change in behavior (Gardner & Stern, 
2002; Kahlor & Rosenthal, 2009). Some type of 
intervention may be necessary to propel individuals 
from beliefs to behavioral changes due to the gap 
between knowing about climate change and changing 
behavior to make a difference (Blake, 1999; Kollmuss & 
Agyeman, 2002). Redman and Redman (2017) found 
subjective knowledge (values, attitudes, and beliefs) to 
be the most important type of knowledge for an 
educational intervention in achieving behavioral goals 
toward sustainability. The affective dimension of 
climate change is believed to provide a critical link 
between knowledge and attitude versus action 
(Leiserowitz, 2006; Sundblad, Biel, & Garling, 2007; 
Weber, 2010). There may be a disconnect between 
attitudes, intentions, and actions due to inadequate 
practical knowledge on how to individually reduce 
emissions (Wolf & Moser, 2011). Individuals need 
specific information on ways in which they can realize 
changes in their energy consumption (Wolf & Moser, 
2011). When students feel like the issue of climate 
change is serious but solvable, there is a positive 
association with pro-environmental behavior (Ojala, 
2012). 


A substantial amount of international research also 
demonstrates that children and young people are more 
likely to understand, care and act on climate change if 
they can engage with it directly and experientially, 
through some form of educational, outreach, or social 
activity. (Corner et al., 2015, p. 528) 
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Therefore, it is important to introduce the issue of climate 
change to students through hands-on, real-world learning 
experiences. 

Hands-on Active Learning 

The strategies used for active learning closely align with 
guidelines employed by the National Research Council to 
develop the Next Generation Science Standards (NGSS) 
(National Research Council, 2012). Inquiry-based science 
was highlighted by the NGSS as the core of teaching and 
learning for students to develop knowledge and 
understanding of scientific ideas including an understanding 
of how scientists view the natural world. The term, inquiry- 
based science, was derived from “hands-on,” “minds-on,” 
and “scientific inquiry” through iterations of science 
education standards (National Academies Press, 2013). 

Student-centered active learning has been shown to 
improve long-term knowledge retention and deep 
understanding (Akinoglu & Tandogan, 2007; Bonwell & 
Eison, 1991; Gallagher, 1997) as well as fostering creative 
and problem-solving skills (Shieh & Chang, 2014). Hands- 
on learning has been shown to positively impact student 
understanding of science concepts (Costu, Unal, & Ayas, 
2007) more effectively than traditional instruction (De la 
Hozi Casas & De Blas del Hoyo, 2009; Sadi & Cakiroglu, 
2011). Studies have also shown hands-on learning activities 
to improve students’ understanding of climate change 
(Karpudewan, Roth, & Abdullah, 2015; Taber & Taylor, 
2009). When using the active learning approach, education 
becomes more personally meaningful and takes advantage 
of students’ natural curiosity. This approach prepares 
students for the future by having students communicate, 
collaborate, and try new approaches in finding solutions to 
real-world problems. A study to find best practices for 
teaching middle school science found that one of the best 
practices is hands-on activities that were often in the form 
of guided science inquiries (Oliveira et al., 2013). Research 
has also shown these types of hands-on, active learning 
strategies increase middle school-aged girls’ interest in 
science as well as have an impact on their ideas about 
participating in a science-related career (Kim, 2016). 

Middle school is an appropriate age to develop an interest 
in science that will persist through secondary school, into 
college and beyond into a career. Providing authentic, 
active learning experiences contributes to the internalization 
of learning about science (Christensen & Knezek, 2015a). 
Teenage years are a good time to introduce the impact of 
climate change when views are still being formed (Corner 
et al., 2015). 

Active learning principles are rooted in Dewey’s 
“learning by doing and experiencing” principle (Dewey, 
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1938). Dewey advocated that a child’s schoolwork should 
have meaning and be engaging as well as have connections 
to other disciplines and life experiences. The curriculum 
used for the participants in this study includes hands-on 
learning activities in which students engage in finding 
solutions to a real-world problem—climate change. 

The purpose of the research study reported in this paper 
was to confirm the appropriateness of behavioral theories 
used to predict an individuals’ readiness to perform a given 
behavior (Ajzen, 1985) to the specific context of climate 
change beliefs and intentions. Through a curriculum that 
provides a comprehensive study of the issues related to 
climate change and tools to see how they can make an 
impact on the reduction of standby power, students can 
realize specific ways in which they can make a change in 
the environment. Leaming principles related to active, 
hands-on learning applied to student beliefs and intentions 
regarding climate change led to hypothesizing an impact in 
this context. 


Research Questions 

This study examined pre-post changes in climate change 
beliefs and intentions among middle school students 
participating in hands-on curriculum activities related to 
energy and climate. Three specific research questions were 
addressed: 

1. To what extent do middle school students who 
participate in hands-on energy monitoring/greenhouse gas 
projection activities become more positive in: (a) beliefs 
that climate change is happening in our world, and (b) 
intentions to take actions that make a difference in reducing 
contributors to climate change? 

2. What gender differences exist for the effects of 
participating in hands-on energy monitoring curricular 
activities, regarding student beliefs about the existence of 
climate change and their intentions to take actions to 
improve the environment? 

3. To what extent are climate change beliefs related to 
climate change intentions? 


Methods 

Instrumentation 

The climate change attitude survey (CCAS) was used in 
this study to measure middle school student beliefs about 
climate change and also their intentions to enact positive 
change. The 15-item survey contains Likert-type items 
rated on a 5-point scale from Strongly Disagree (1) to 
Strongly Agree (5). CCAS was developed by the authors 
based on adaptations from other climate-related surveys 
intended for different audiences or different purposes 
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(Champeau, 1997; Leiserowitz et al., 2013). The instrument 
viewed as a whole was found to be reliable for middle 
school students (Cronbach’s Alpha = .89 for 15 items) and 
construct validity was determined through factor analysis 
(Christensen & Knezek, 2015b). Two constructs were 
represented among the items: Beliefs and Intentions 
(Christensen & Knezek, 2015b). The current study’s set of 
data revealed respectable reliability as measured by 
Cronbach’s Alpha (.90 for Construct 1 and .78 for 
Construct 2). Construct 1 consisted of 10 items related to 
beliefs and included items such as “I believe there is 
evidence of global climate change.” Construct 2 consisted 
of five items related to intentions, such as “We cannot do 
anything to stop global climate change” (reversed). 
Participants 

The CCAS was completed by middle school students in 
grades 5-8 participating in the Going Green! Middle 
Schoolers Out to Save the World (MSOSW) project. For 
the analyses presented in this paper, pre-post data were 
collected from 794 treatment and 521 comparison students 
from 18 treatment classrooms and 7 comparison 
classrooms. These students represented schools in the states 
of California, Hawaii, Louisiana, Maine, Michigan, North 
Carolina, Texas, Vermont, and Virginia. The school 
locations were selected for their diversity in climate zones, 
rural versus urban locations, and public versus private 
funding status. The gender representation included 53.6% 
boys and 46.4% girls. The largest percentage of students 
(47%) were in 6th grade due to the better fit of the 
curriculum to that grade level in most states. Students were 
asked to select the ethnicity to which they most identified 
and the data included 50.7% Caucasian, 14.6% Asian, 
12.8% Hispanic, 7.4% African American, 3.5% Native 
American/Pacific Islander, and 11.1% Other. Fifty-two 
percent of the students represented urban areas while 48% 
represented rural schools. The majority of students were 
from public schools (89%). Regarding climate zones, 
students were from 6 of the 12 different climate zones in the 
United States: East North Central, Northeast, South, 
Southeast, West, and South Pacific. These demographic 
data, along with other attitudinal and content data related to 
STEM and demographic items, were gathered through an 
online server at the beginning (pre) and end (post) of the 
2015-2016 school year, as part of the project’s annual data 
collection activities. Teachers who attended the summer 
institute received instructions on administering the surveys 
while all teachers were sent instructions via email from 
project personnel. Students either went to a school lab or 
had access to individual laptops to submit data under the 
supervision of their teachers. 
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About the Project 

The Going Green! MSOSW project aims to direct middle 
school students’ enthusiasm for hands-on activities toward 
long-term interest in STEM while guiding them to solve 
real-world problems. The study included both treatment 
and comparison groups of students. Treatment classrooms 
were selected by inviting teachers via colleagues in multiple 
locations from different climate zones in the United States. 
Comparison classrooms were selected from those who 
expressed interest in the project but were not initially 
included due to limitations of numbers of classrooms or the 
classroom teachers were invited as similar comparisons of 
those who participated. The goal was to enhance 
curriculum topics that were already in place as opposed to 
an “add-on” to curriculum. The comparison classrooms 
collected pre-post data but taught the normal curriculum on 
climate change provided by their schools, which followed 
their own state or national standards. Beginning in the year 
after they provided comparison data, teachers were invited 
to be treatment classrooms for the project. 

In the MSOSW project, treatment classroom teachers 
attended a two-day summer institute to learn about the 
energy/climate curriculum and how to implement the 
hands-on activities with their students. MSOSW teachers 
were also provided with classroom sets of 30 energy 
monitors, web enhanced teaching resources, the curriculum 
itself, and on-going support from the project personnel. 
Students in this project are taught by their teachers to use 
energy-monitoring equipment to audit standby power 
consumed by electronic devices in their homes and 
communities. The focus of the three-week curriculum and 
activities is standby power, which is power that is being 
used by appliances when they are plugged in but serving no 
useful function, and how this waste contributes to 
greenhouse gas emissions and global climate change. The 
meters used for the monitoring activities can be adjusted for 
local electricity costs and have single-button calculations of 
the connected appliances’ consumption of kilowatt hours, 
cost per month, and CO) released into the atmosphere while 
generating the electricity consumed in standby mode. 

Standby power is the electricity consumed by many 
appliances when they are plugged in but “turned off’ (U.S. 
Department of Energy, 2011). The U.S. Department of 
Energy has estimated that over the lifetime of a typical 
home appliance, 70% of the power consumed will be when 
the appliance is turned off (U.S. Department of Energy, 
2011). 

The curriculum includes hands-on, engaging activities in 
which the goal is to have students understand and 
internalize the impact they can individually and collectively 
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have on the climate by doing simple things like unplugging 
devices in their homes and schools that were plugged in but 
serving no useful function (standby power). The activities 
included researching and comparing types of renewable and 
non-renewable energy sources used to power their 
electricity, hands-on measurement of energy consumption, 
greenhouse gas experiments, and using a simulated online 
game environment. Each of the lessons in the unit includes 
a list of the educational standards addressed, such as the 
NGSS. For example, in lesson two when students are 
identifying evidence and possible causes of climate change, 
they are addressing science as inquiry asking questions to 
clarify evidence of the factors that have caused the rise in 
global temperatures over the past century (NGSS MS- 
ESS3-S). 

During MSOSW project activities, middle school 
students learn to measure the standby power consumption 
of various appliances in students’ homes. Students produce 
a home inventory of appliances they believe might be 
consuming standby power, then take an energy monitor 
home after being trained by their teacher. The students 
monitor home appliances, record data, and then take their 
recorded measures to school where the observations of all 
students are combined into a spreadsheet projection model 
produced under the guidance of the teacher. After 
measuring standby power, students aggregate their data for 
spreadsheet projections to explore energy conservation 
plans that could lower a family’s monthly electric bill and 
reduce the greenhouse gas emissions that contribute to 
global climate change. Classes use the spreadsheet models 
to perform “what if’ projections of the effect of unplugging 
vampire (standby) power consuming devices, and 
determine the effect of their findings on the changing 
climate in our world. Students share their results with other 
participating middle school students from across the United 
States by contributing to an online project wiki in which 
other classrooms in the project can see and compare their 
findings to other classrooms participating in the project. 
Students also shared their public service announcements on 
the project wiki. 


Data Analysis and Results 

Pre-Post Changes in Climate Change Beliefs and 
Intentions 

The CCAS was used to assess two constructs regarding 
climate change—beliefs in climate change and intent to 
take action based on those beliefs. One-way analysis of 
variance (ANOVA) was conducted to compare differences 
between treatment and comparison groups at pretest and at 
posttest for the two constructs. Additional ANOVAs were 
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Table 1 
Total Climate Change Survey, Pre-Post, Treatment versus Comparison 
N Mean SD Sig. ES 

ee ror ee 
Treatment Pre 792 3.65 54 

Postay 733 3.86 56  ~—-.0005 38 
Comparison Pre 508 3.56 ay. 

Post o2k 3.66 .60 005 eli] 





conducted to compare girls and boys at pretest and posttest. 
Pearson r correlation analyses were conducted to determine 
the relationship between the climate change constructs 
between pretest and posttest time for both treatment and 
comparison participants. 

Table 1 shows the treatment and comparison pre-post 
means for all 15 items on the survey. One-way ANOVA 
was used to compare pretest to posttest for both treatment 
and comparison students. Effect sizes were calculated to 
appraise the educational significance (Bialo & Sivin- 
Kachala, 1996) of the findings in addition to the statistical 
significance. 

Regarding Research Question #1, while both treatment 
and comparison groups had significant increases (p < .05), 
the increase was stronger for the treatment group as is also 
shown by the effect sizes for each group (ES = .38 for 
treatment and ES = .17 for comparison participants). 

For the 792 students in the treatment classrooms across 
all grade levels, there were significant (p < .05) positive 
changes in both their beliefs (p < .0005) (ES = .43) and 
intentions (p< .05) (ES = .20) to make changes to help 
address climate change issues (see Table 2). For the 508 
students in the comparison classrooms, there were also 
significant increases in positive beliefs (p = .02) (ES = .15) 
from pretest to posttest time periods. For intentions, the 
comparison group gains were significant (p < .05) as well 
(p = .046) (ES = .21) (Table 2). As shown in Figures 1-3, 
the gains for the treatment group in climate change beliefs 
(ES = .43) were much larger than for the comparison group 
(ES = .15), with the treatment group gaining significantly 
(p<.05) more than the comparison group regarding 
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Figure 1. Climate change beliefs, pre-post, treatment versus comparison 
students. [Color figure can be viewed at wileyonlinelibrary.com] 


climate change beliefs. There were no_ significant 
differences between the treatment and comparison groups 
for gains in climate change intentions. 
Disaggregated Findings by Individual Items 

Because of the focus of the project on specific actions 
that students could take to reduce CO, production and its 
impact on our climate, an item analysis of the CCAS was 
conducted. For the treatment group all of the individual 
items increased (binomial p < .0001) with 12 of 15 items 
having significant (p < .02) increases individually. For the 
comparison group, 10 of the 15 items increased but this 
occurrence was not found to be rare by chance (binomial 
p=.15). The three of the 15 items that had significant 
(p < .02) increases pre to post for treatment students but not 
for comparison students were: “I am concerned about 
global climate change”; “The actions of individuals can 
make a positive difference in global climate change”; and “I 
think most of the concerns about environmental problems 
have been exaggerated (reverse coded).” The one item that 
had a significant (p = .03) increase for the comparison 
group but not the treatment group was “‘It is a waste of time 
to work to solve environmental problems” (reverse coded). 

Three additional items were added to the original two- 
construct survey instrument to pilot test their abilities to 
enhance the original constructs. Of the three items, one 
showed significant differences between the groups and was 





Table 2 
Pre-Post Differences for Treatment and Comparison Students for Two Climate Change Constructs 
Treatment Comparison 

N Mean SD Sig. ES N Mean SD Sig. ES 
Pre Climate change beliefs 792 3.76 .67 508 3.67 70 
Post 733 4.04 .63 .0005 43 521 3.78 76 021 LS 
Pre Climate change intentions 792 3.48 75 508 B37 76 
Post 7133 3.63 19 .0005 .20 521 553 74 001 bd] 
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Figure 2. Climate change intentions, pre-post, treatment versus comparison 
students. [Color figure can be viewed at wileyonlinelibrary.com] 


a significant discriminator for the treatment versus 
comparison group. For the item, “Environmental problems 
can be solved without big changes to our way of life,” there 
was a large difference between treatment students and 
comparison students. Students who participated in the 
standby power monitoring activities increased significantly 
(p = .03) while the comparison students actually decreased 
(although NS). The other two additional items showed no 
significant differences between treatment and comparison 
students. 
Effect of Gender on Climate Change Beliefs and 
Intentions 

Additional analyses were conducted on the data 
disaggregated by girls and boys to determine whether 
changes in beliefs and intentions attributable to project 
activities differed for boys and girls. While both climate 
change beliefs and intentions were examined for pre-post 
gains, there were only significant differences (p < .05) for 
climate change beliefs. As shown in Table 3, the treatment 
group girls gained significantly (p = .009, interaction term 
f= 6.81, df= 1) more than the comparison group girls on 
climate change beliefs. The magnitude of the effect of 
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Figure 3. Treatment versus comparison pre and post for climate change 
beliefs (p<.05) and intentions (NS). [Color figure can be viewed at 
wileyonlinelibrary.com] 
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Table 3 
Treatment/Comparison by Pre-Post Means Regarding Climate Change 


Beliefs for Girls and Boys 





Girls Boys 
N Mean SD N Mean SD 
Treatment pretest 358 43.13 86,00 OG 63.18 ole 
Comparison pretest... 25,1 03.69, O82 25 Ldn nail 
Treatment posttest 331 4.04 58 402 4.04 .67 
Comparison posttest’ 256 3:80) 660265, 3:75 4) 71 





MSOSW treatment group activities over the comparison 
group was moderate (Cohen’s d= .33) and educationally 
meaningful according to established guidelines (Bialo & 
Sivin-Kachala, 1996). As shown in Table 3, for boys, the 
effect of MSOSW activities on climate change beliefs for 
the treatment group was also significantly (p =.05, 
interaction term= 3.82, df=1) greater than for the 
comparison group. The magnitude of the effect of treatment 
versus comparison was small (Cohen’s d= .22). As shown 
in Figures 4 and 5, these findings collectively indicate that 
the positive effect on girls was stronger than the effect on 
boys regarding climate change beliefs. These findings align 
with results in prior MSOSW project years regarding 
gender differences—that the effects were especially strong 
for girls (Knezek, Christensen, Tyler-Wood, & Gibson, 
2015). 
Relationship of Climate Change Beliefs to Intentions 
Additional analyses were conducted to examine the 
relationship between climate change beliefs and intentions 
(RQ3). As shown in Table 4, at the time of the pretest for 
treatment students, climate change beliefs and intentions 
were significantly correlated (r = .307, p < .0005) with a 
moderate strength of relationship (Cohen, 1988; Lenhard & 
Lenhard, 2016). Nine percent (9%) (.307 X .307) of their 
variance was in common. Also shown in Table 4, for the 
treatment group at the time of the posttest, the strength of 
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Figure 4. Treatment students male versus female for climate change beliefs 
and intentions. 
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Figure 5. Comparison student male versus female pre-post for climate 
change beliefs and intentions. 


the association between climate change intentions and 
climate change beliefs had increased to r=.427 
(p < .0005) with 18% (.427 X .427) of their variance in 
common. This represents an increase from a moderate 
positive relationship between beliefs and intentions before 
MSOSW activities, to a large positive relationship after 
MSOSW activities. 

As shown in Table 5, at the time of the pretest for 
comparison students, climate change beliefs and intentions 
were significantly correlated (r= .312, p< .0005) but the 
strength of the relationship was moderate (Cohen, 1988; 
Lenhard & Lenhard, 2016) with 10% (.312 X .312) of their 
variance in common. Also shown in Table 5, for the 
comparison group at the time of the posttest, the strength of 
the association between climate change beliefs and climate 
change intentions remained almost identical at r= .317 
(p < .0005) with 10% (.317 X .317) of their variance in 
common. The moderate strength of association between 
climate change beliefs and intentions remained unchanged 
for the MSOSW comparison group, from pre to post. 

As an answer to Research Question # 3, these findings 
provide evidence that a moderate (Cohen, 1988) positive 
relationship exists between climate change beliefs and 
intentions among middle school-aged students across the 


United States in their normal school environments, and that 
hands-on energy monitoring activities such as_ those 
provided by the Middle Schoolers Out to Save the World 
project can strengthen the association between the two 
constructs, from 9 to 18% of their variance in common. 
This is an increase from a moderate relationship to a 
moderately large relationship according to guidelines 
provided by Cohen (1988) of .1 = small, .3 = moderate, 
and .5= large for Pearson Product Moment correlation 
coefficients. This is important because according to the 
theory of planned behavior (Ajzen & Fishbein, 1980), 
positive attitudes lead to a higher intention to perform a 
related behavior. In this case, the related behavior is intent 
to take action to reduce climate change. 


Discussion 

One purpose of this study was to measure the changes in 
climate change beliefs and intentions for students who 
participated in a hands-on, active learning curriculum in 
which students studied climate change and used take-home 
meters to measure standby power. By measuring the amount 
of power that was being wasted when appliances were 
plugged in yet serving no useful function, students were able 
to conclude that they could individually as well as collectively 
make a difference in the amount of CO, in the atmosphere. 
This kind of project is likely to have the type of impact 
reported by Corner et al. (2015) regarding youth engagement 
in climate change issues, due to the treatment students directly 
and experientially engaging with an educational activity 
regarding climate change that leads to action. 

This study sought to address the disconnect between 
knowledge and attitudes versus action reported by many 
researchers (Leiserowitz, 2006; Sundblad et al. 2007; 
Weber, 2010) by following the principles of the theory of 
planned behavior (Ajzen & Fishbein, 1980) in which 
positive attitudes lead to a higher intention to perform a 
related behavior. It is also an attempt to address the 





Table 4 
Correlation of Climate Change Belief and Intention Factors at Pretest and Posttest Time for Treatment Students 
Pretest Posttest 
Belief Intentions Belief Intentions 

Climate change belief factor Pearson correlation ] SO i7% 1 427** 

Sig. (two-tailed) .000 .000 

N 792 792 733 qe 
Climate change intentions factor Pearson correlation Oda. 1 427** 1 

Sig. (two-tailed) .000 .000 
N 792 792 733 132 


Note. **Correlation is significant at the .01 level (two-tailed). 
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Correlation of Climate Change Beliefs and Intentions at Pretest and Posttest Time for Comparison Students 


Pearson correlation 
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Note. **Correlation is significant at the .01 level (two-tailed). 


suggestion that individuals need specific experiences in 
ways they can realize change in order to take action (Wolf 
& Moser, 2011). In a study related to beliefs, Pajares (1992) 
concluded that “beliefs are the best indicators of the 
decisions individuals make throughout their lives” (p. 307). 
Dewey’s “learning by doing and experiencing” principles 
were applied in this study to use hands-on learning 
curriculum to engage students in finding solutions to real- 
world problems to which they have experienced success in 
participating in making changes. Changing beliefs through 
hands-on real-world activities can likely change the 
intentions and behaviors of students who come to the 
conclusion that they can make a difference as an individual 
as well as a collective group. Recording a measurable 
increase in level of intentions for a specific group due to 
increase in beliefs may require waiting some time. For 
example, one school year cycle may be a reasonable 
amount of time lag to expect that strengthened beliefs in 
climate change might migrate through a learner’s affective, 
reinforcing, and directive behavioral repertoire (Staats, 
1975) to result in a measurable increase in intentions. 
Additional research is needed in this area. 


Table 6 
Comparison of Cohen's d and Pearson r Effect Sizes 


i Cohen (1988) 


.00 No effect 
05 

10 Small effect 
lS 

2. Zone of desired effects 
.24 Intermediate effect 

29 

33 

.37 Large effect 
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Hattie (2009) 
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Note. Adapted from Lenhard and Lenhard (2016). 
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The findings from this study with respect to gender 
indicate a stronger positive effect for girls than for boys 
regarding beliefs in climate change. This may be due to 
females feeling a stronger affinity for improving and 
making a difference in the world (Modi, Schoenberg, & 
Salmond, 2012). This type of learning environment 
provides interactions among students and provides a social 
learning context that females often prefer (Jacobs, Kuriloff, 
Andrus, & Cox, 2014) as well as providing connections 
between school and the real world. 

Future research is planned to update the survey 
instrument with the additional three items and run factor 
analysis on various groups of participants. It is planned to 
use this data to compare the beliefs and intentions of the 
middle school students who were the focus of this study, to 
those of teachers and other STEM professionals. 

In the broadest context, the research reported in this 
paper served as a test of whether the study of climate 
change beliefs and intentions can be anchored within 
existing behavioral theories used to predict an individuals’ 
readiness to perform a given behavior (Ajzen, 1985). Initial 
indications are promising, as established through the 
confirmation of significant (p < .05) positive correlations 
between climate change intentions and beliefs, with 
stronger correlations emerging for students who have taken 
part in energy monitoring activities. However, correlation 
does not establish causation, so further research is needed in 
this area. It is possible that beliefs cause intentions, 
intentions cause beliefs, or something else causes both. 

There are additional limitations to this study other than the 
causation issue reported in the previous paragraph. The 
subjects in this study were the students of teachers who 
volunteered to participate in a project to monitor unnecessarily 
wasted electricity and find ways to possibly reduce climate 
change. Therefore, these students’ attitudes may have been 
influenced by their teachers, regardless of whether the teachers 
represented a treatment site or comparison site at the time of 
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acquisition of data. Also a large number of natural disasters 
occurred in the United States during the year (2015-2016) 
when this study took place. These events were in the public 
eye and may have influenced the treatment as well as 
comparison group students. 


Summary and Conclusions 

The findings presented in this paper provide evidence that 
curriculum projects with hands-on learning activities have a 
positive influence on climate change beliefs and intentions, 
and increase the strength of association between the two 
constructs. In this study, for the treatment group of MSOSW 
project students, climate change beliefs and intentions 
moved from a moderate positive relationship between 
beliefs and intentions before project activities, to a strong 
positive relationship after project activities. The comparison 
group remained largely unchanged. Both treatment and 
comparison groups became more positive in their climate 
change beliefs and intentions, pre to post, and in the case of 
beliefs the gains for treatment students were significantly 
larger than for the comparison students. Regarding gender 
differences in impact of treatment, females showed a larger 
positive change regarding climate change beliefs. These 
findings are similar to previous results reported by the 
authors on a different set of data (Christensen, Knezek, & 
Tyler-Wood, 2015) as well as findings from other similar 
studies (Redman & Redman, 2017). 

Implications of these findings for educational practice 
include having students connect to real-world problems by 
participating in activities where they can make a small 
difference individually but a large improvement 
collectively. The impact of changes in beliefs and intentions 
for students in middle school grades is large considering 
that the impact can influence students for the rest of their 
lives, guiding them in the direction of making thoughtful 
decisions in their daily lives. 
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STEM education in elementary school is guided by the understanding that engineering represents the application of 
science and math concepts to make life better for people. The Engineering Design Process (EDP) guides the application 
of creative solutions to problems. Helping teachers understand how to apply the EDP to create lessons develops a 
classroom where students are engaged in solving real world problems by applying the concepts they learn about science 
and mathematics. This article outlines a framework for developing such lessons and units, and discusses the underlying 
theory of systems thinking. A model lesson that uses this framework is discussed. Misconceptions regarding the EDP that 
children have displayed through this lesson and other design challenge lessons are highlighted. Through understanding 
these misconceptions, teachers can do a better job of helping students understand the system of ideas that helps engineers 
attack problems in the real world. Getting children ready for the 21st century requires a different outlook. Children need 
to tackle problems with a plan and not shrivel when at first, they fail. Seeing themselves as engineers will help more 


underrepresented students see engineering and other STEM fields as viable career options, which is our ultimate goal. 


The value of an education heavily steeped in science, 
technology, engineering and math (STEM) has been 
advertised in the popular media as well as in education 
circles. Unfortunately, large-scale testing of school-aged 
children’s science and mathematics understanding, like the 
Program for International Student Assessment (PISA) and 
the Trends in International Mathematics and Science Study, 
have shown that the United States lags behind other 
industrialized nations (Ginsburg, Cooke, Leinwand, Noell, 
& Pollock, 2005). Furthermore, the testing focus on reading 
and mathematics of the current political environment has 
edged out science and social studies from elementary 
classrooms (Walker, 2014). Careers in STEM fields are the 
ones that not only will be most lucrative, but will be 
necessary in the coming years and decades as more and 
more jobs float overseas, leaving the blue-collar workers in 
a chasm with regard to opportunities to hold a job to sustain 
a family. There will be an increasing need for STEM 
education to provide career paths and opportunities in a 
U.S. marketplace competing globally (Shilling, 2015). 
STEM careers represent the jobs that will be needed to 
continue to support innovation and change as we continue 
the fast forward into the 21st century (Trilling, Fadel, & 
Partnership for 21st Century Skills, 2009). The educational 
design espoused in the current project has been developed 
and rationalized via the genesis of research in STEM 
educational design and learning theory as guided by our 
review of the literature. Thus, the concurrent validity of our 
approach and the design we articulate is based upon our 
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adoption of research-based STEM design frameworks 
technically evaluated for educational performance (Berland, 
2013). 

Engineering represents the application of science and 
mathematics concepts to make life better for people. This 
definition is the connecting glue for elementary-aged 
students and answers the inevitable question, “How am I 
ever going to use this?” Learning the concepts of mixtures 
and solutions in science and volume and capacity in 
mathematics ties directly to the human need for clean water. 
Communities around the world struggle with providing 
clean water for residents, including children. Research by 
the Girls Scouts Research Department indicates that in 
elementary school, girls are interested in STEM fields, and 
even can see themselves as holding a STEM career as 
adults (Modi, Schoenberg, & Salmond, 2012, p. 25). Even 
in middle school, girls indicate they like STEM fields, but 
other non-STEM career options become more prominent as 
they get older and work their way through high school. 
Health fields such as nursing and dental hygiene remain on 
young women’s career paths, but engineering fields, 
mathematics fields, and even technology fall to the wayside 
as women choose fields where they perceive they will be 
able to help people and the world (Modi et al., 2012, pp. 
25-27). Furthermore, developing an interest in STEM early 
is crucial. For males and females, according to Sadler et al., 
the interest students have upon entering high school is the 
best predictor of their career interests upon leaving high 
school (Sadler, Sonnert, Hazari, & Tai, 2012). The 
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challenge, then, is to help young women, along with all 
students of underrepresented populations, to see how 
STEM fields outside of the medical fields do help humanity 
and the environment. 

Children’s misconceptions about science concepts have 
theoretical underpinnings. Posner et al. propose two sets of 
knowledge, a child’s naive understandings and the adult’s 
correct understanding about scientific phenomena. 
Children, when confronted with the disconnect abandon 
their naive beliefs and reconceptualize their understanding 
according to the adult view (Posner, Strike, Hewson, & 
Hewson, 1982). Piaget even wrote about children 
experiencing disequilibrium when confronted with 
information that challenges their incorrect understandings, 
with the incorrect beliefs displaced by the new more mature 
understandings (Piaget, 1977). We will discuss children’s 
misconceptions that were discovered as a result of using the 
lesson that is presented. 

Helping elementary school teachers understand the 
pedagogy of how to connect engineering with science and 
mathematics and to use inquiry environments in new and 
creative ways is the first step to meet this challenge. Within 
this article, we share our conception of how to develop an 
understanding of the Engineering Design Process (EDP) 
along with a model for lesson design format teachers can 
use to create design challenges for their own units and 
lessons. Understanding the EDP will be developed through 
the model of an original lesson plan and the discussion of 
children’s misconceptions of the elements of the design 
process. 


EDP and STEM Education 

According to Lammi, “Engineering design is more than 
the mere manipulation of numbers and the solving of 
scientific equations. The processes employed in engineering 
design encompass a broad variety of topics and fields of 
study” (Lammi, 2011, p. 3). In 1973, a professor at the 
University of Negev, Israel, created a short but 
revolutionary document on the “Design Process.” Dr. 
Berard Roth described an often-occurring problem, that 
engineers may come too quickly to a solution, only to find a 
better solution with further thought. In this document, he 
outlined a Design Process, describing how successful 
engineers go through a series of steps to critically think 
about the problem and as a result, create alternative 
approaches of solutions. “It is a series of steps through 
which the design passes before it is completed. By making 
this a conscious process, the engineer can greatly improve 
his chances of arriving at a better solution” (italics in 
original) (Roth, 1973, p. 4). This unassuming start is the 
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F See. for an Engineering Design Challenge Module 
Framework 

Lesson Title Fill In 
Literature opening Title of book: 


Focus in the book: 


Curriculum Standards Source of Standards: 


Science Inquiry Lesson 1: 
(attach lesson plans) Lesson 2: 
Introduction of Engineering Design Challenge: 


Design Challenge 
Sharing strategy 





foundation for the EDP. A critical outcome of facilitating 
student understanding of the EDP is that students engage in 
systems thinking where multiple viewpoints and elements 
are conceptualized toward problem solving (Silk & Schunn, 
2008). The lesson model framework we propose 
incorporates a model for incorporating the EDP, while 
enabling students to overcome science misconceptions. 
Lesson Model for Engineering Design Challenges 

Introducing children to the idea of engineering, the 
application of science and mathematics to make life better 
for people, helps to frame the lessons involved in an 
engineering design challenge module. Within the idea of a 
“module,” teachers can structure science and math lessons 
that build concepts students can use to complete a design 
challenge. Ideally, a module would be a series of lessons on 
concepts students have some foundational knowledge. 
However, a module can be done in one day, or as a series of 
lessons culminating in a more complex design challenge. A 
design challenge is a “problem task” that students use the 
EDP to find a workable solution. The following will be a 
discussion to illustrate all of these ideas. 
The Engineering Design Challenge Module 

The Museum of Science, Boston has developed a series 
of engineering kits, called Engineering is Elementary (EiE) 
program kits, that models a very nice application of science 
and mathematics into a challenge for young engineers 
(http://eie.org/overview/engineering-design-process). Their 
model is one that is worthy of being a framework for the rest 
of us to use to develop our own curriculum (see Table 1). 
The use of this curriculum as a foundation for our model 
has a strong foundation in research. Student motivation, 
engagement, and enjoyment of science was seen to 
increase when instruction incorporated the EDP 
(Macalalag, Lowes, Guo, McKay, & McGrath, 2009). 
Further, teachers reported students of underrepresented 
and racial minorities in science showed improved 
performance (Engineering is Elementary Research 
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The Engineering Design Process 





The Goal 






Create | 


Figure 1. The EDP diagram from EIE. 
Source: Teach Engineering Curriculum for K-12 Educators/Engineering 
Design Process 2003. 


Department, 2014). Engaging students with a STEM 
curriculum illustrates how engineers apply science, 
technology and mathematics knowledge to solve problems 
(Sedberry, 2014), and integrated approaches to learning 
provide “more relevant, less fragmented, and more 
stimulating experience for learners” (Furner & Kumar, 
2007, p.186). Finally, in research on teacher professional 
development by the Program for International Student 
Assessment (PISA), a significant correlation was found to 
exist between gains in science content knowledge and the 
number of engineering design activities students engaged in, 
showing a positive effect for the use of classroom 
curriculum with foundations in the EDP (Macalalag & 
Tirthali, 2010). 

The framework starts with a literature link. The use of 
children’s literature can help to provide a “real life” 
application for the concepts with the use of the story line. 
After a defining of the curriculum standards, the module 
moves into an inquiry activity with science and/or math 
concepts that will support the ultimate development of the 
design challenge. After discussion of the investigation, 
students are challenged to solve the problem that was 
introduced through the literature link. This is the actual 
jump into the EDP, illustrated in Figure 1. 

EDP: How It Works 

The EDP process, which takes on various forms 
depending on the publisher you choose, is the foundation of 
the process actually used by engineers. It consists of a series 
of steps that guide engineers as they solve problems (www. 
teachengineering.org/k12engineering/designprocess). One 
example from EiE is simple and an easy-to-follow model 
(Figure 2) (http://www.eie.org/). Almost always, the model 
is circular and does not indicate a particular starting point. 
This is because an engineer might jump into the process at 
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Figure 2. Examples of successful carrot Oscars. 


any point, and although one generally goes through the 
steps in a clockwise fashion, even this is not a requirement. 
The model of a circle is important in that engineering is not 
a linear activity meant to reach the goal and be done, but a 
constant back and forth of questioning, creating and 
improving. Table 2 explains the steps in the design process, 
and illustrates each step with an example for a real product 
many of us use. A product may have a design flaw or 
something that needs improvement, which would lead an 
engineer to start with the improve step. This would be 
instead of starting at Ask, which would be where an 
engineer solving a problem for the first time would 
probably start. In the lesson that follows, the focus was 
specifically on the plan and create steps, getting children to 
not just jump into making something without thinking 
about it first, or without considering options. Teams had to 
submit their plan, complete with a materials list prior to 
getting materials, and being able to make a prototype. 
Analyzing and Interpreting Data 

A key component to the planning and implementation of 
this lesson is to provide children with an opportunity to 
collect data that can then be analyzed to use for creating a 
solution for the structured problem. This practice, Analyzing 
and Interpreting Data, is one of the key components of the 
NGSS Science and Engineering Practices (NGSS, 2013; 
Willard, 2014). Children in primary grades should be able to 
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Table 2 

Engineering Design Process (EDP) 

EDP Step Definition Example 

Improve A product has an aspect that is problematic. The travel cup lid has a vent hole that drips coffee on 

the floor of the car. 

Ask Define an engineering type of problem. How could the hole be closed so drips don’t leak out? 

Imagine Brainstorm ideas. Come up with several ideas. 

Plan Choose the best idea and sketch it out, make a list Develop the idea of putting the hole in the slide that 
of materials, etc. closes the drinking hole. 

Create Make a prototype. Make the prototype and test it—it works! 

Improve Is there a problem? My car carpet is drip free. 





test according to an attribute—in this case, does it float?— 
then record the data into a chart (Willard, 2014, p. 53). 
Teachers should be able to identify this step as inquiry-based 
and constructivist in nature, creating an experience for 
students to be hands-on with the available materials, testing 
their floating qualities, and proving for themselves what floats 
and what does not along with potential reasons why. The goal 
is then to see if children will use the data they gathered in 
creating solutions for the problem. 
Systems Thinking 

In systems thinking, the individual parts are identified as 
well as the understanding of how the parts work together to 
make the whole. “Analysis and design of systems is central 
to the work of engineers as they seek to modify their 
surroundings for particular purposes.” (Silk & Schunn, 
2008, p. 6). Lehrer and Schauble (1998) conducted 
interviews of second- through fifth-graders about gear 
systems that served a purpose, like in an egg beater or a 
bicycle, and gear systems that served no actual purpose. 
They found that second-graders had difficulty being able to 
connect the elemental parts of the system together to 
understand how the gears were working together. Fifth- 
graders were able to create “causal chains” of three or more 
parts, indicating a clearer understanding of how parts work 
together. However, both groups were better able to identify 
components that worked together when there was an action 
as an end result. For example, younger children were able 
to understand the purpose of the gears, and the teeth on 
them that interconnect that leads to the spinning of the 
beaters to whisk the eggs (National Academy of 
Engineering and National Research Council, 2009, p. 123). 
The NGSS Standards include Systems and System Models 
as a Crosscutting Concept (Willard, 2014, p. 53). For K-2 
students, this involves being able to describe an object or 
organisms in terms of their parts, as well as being able to 
identify the parts that work together in the natural world as 
well as the designed world (Willard, 2014, p. 59). 
Engineering design challenges often ask students, even 
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young students like the second-graders in the following 
example, to think about using materials and how materials 
work together to solve the problem. 


Sample Teaching Module 
Literature Opening 

Using the Framework in Figure 1, we will outline a 
module created for second graders (Table 3) To start, we 
used the book, The Hallo-Wiener, by Dav Pilkey (1995). 
This is a story of dogs Trick-or-treating who get scared 
by costumed cats and run into a pond in fear for their 
lives. The underdog hero of the story is a dachshund, 
Oscar, who is wearing the embarrassing hotdog costume 
made by his mom. He hears the cries for help and as he 
waddles up he notices cat feet under the frightful 
costume. Oscar rips the costume off and the cats run 
away in fear. He jumps into the pond and swims to his 
friends who climb aboard his floating hot dog costume 
and Oscar becomes the hero of the night. In this module, 
we focused on the fact that Oscar’s costume floats, and 
what do we know about why things float and sink. 
Although there is a serious breach of anthropomorphism, 
obviously, dogs do not dress up and go trick-or-treating, 
it is such a serious walk into fantasy that children see it as 
such. The further reason to use it is that in one of the 
most common basal reading series there is an author 
study highlighting Dav Pilkey, the author of the Dumb 
Bunnies series, Captain Underpants series, and other 
such favorites as Dog Breath, Kat Kong, and Dogzilla. 
The use of The Hallow-Wiener can tie together class 
work as well as entice teachers to use the engineering 
module. 

In recent years, there has been an increase in the number 
of STEM, engineering and science focused books for 
children. Andrea Beaty has several, such as Rosie Revere, 
Engineer (Beaty, 2013) and Iggy Peck, Architect (Beaty, 
2007). Other books introduce children to others who have 
worked to create something new, or used creativity to solve 
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Table 3 
Model of the Framework in Action 
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SL 


Title 


Oscar Floats! 





Literature opening 


Curriculum standards 


Science inquiry 


Title of book: The Hallo-Wiener (D. Pilkey) 

Focus in the book: Oscar’s costume floats 

Source of Standards: NGSS: Science and Engineering Practices 

Practice 2: Develop a simple model based on evidence to represent a proposed 
object or tool. 

Practice 3: Plan and conduct an investigation collaboratively to produce data to 
serve as the basis for evidence to answer a question, and Make observations of 
a proposed solution to determine if it meets the goal. 

Practice 4: Record information and, analyze data from tests of an object or tool 
to determine if it works as intended. 

Lesson 1: Inquiry: Test sample materials to determine floatability 


(attach lesson plans) Lesson 2: n/a 
Introduction of Engineering Design Challenge 


Sharing strategy 


Design Challenge: Design a “costume” for Oscar that will help him float. 
Museum Walk 





a problem. Zhe Boy Who Harnessed the Wind 
(Kamkwamba & Mealer, 2012) and Marvelous Mattie: How 
Margaret E. Knight Became an Inventor (McCully, 2006) 
are both such books. Then others, like The Hallo-Wiener, 
are stories that inspire a design challenge. Leo Cockroach, 
Toy Tester (O’Malley, 1999) or love flute [sic] (Goble, 
1992) may inspire challenges of flight or making musical 
instruments. 

In planning a module, teachers would need to identify 
what the science concept is that grounds the design 
challenge inspired by the chosen book. Inquiry activities 
are readily available on almost any elementary science 
concept. Teachers may have textbook or kit resources 
in their schools, or they may use websites like 
www.sciencebuddies.org, info.collaborativeclassroom.org, 
or even STEMwise.education.olemiss.edu. The purpose of 
the inquiry needs to focus on developing the foundational 
understanding that children can then apply to the design 
challenge task to be successful. 

The Standards 

Referring to the framework, after the text has been 
identified, we can match the focus from the book to science 
or math standards. Although “floating and sinking” are no 
longer a specific part of the K-2 science curriculum, 
Process Standards from NGSS become the larger focus for 
this investigation. Practice 2 in the NGSS (2013) state that 
K-2 students need experiences in using diagrams and 
drawings as models to represent concrete design solutions 
(NGSS, Appendix F, p. 6). Specifically, students need to 
have opportunities to “Develop a simple model based on 
evidence to represent a proposed object or tool” (NGSS 
Appendix F, p. 6). This also ties in nicely with Practice 4, 
Analyzing and Interpreting Data, where students are to 
record information to identify patterns in the natural and 
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designed world to solve problems. Students should be 
presented with opportunities to gather data to analyze and 
use in the construction of the solution strategy. Finally, 
students should “‘analyze data from tests to determine if it 
works as intended” (NGSS Appendix F, p. 9). 

These process standards fit into the aspects of the EDP in 
this teaching module in the PLAN stage with “creating a 
model” (in this case a drawing/sketch), CREATE and 
IMPROVE steps. When students are planning, we want to 
see if they use the data from their prior tests of materials, 
analyzing materials that would be best and including them 
into the plan? Do the materials used work together as a 
system? Do the parts work together to make a workable 
solution? When they create, does it have anything to do 
with their plan? After they test it, do they understand that if 
Oscar sinks, that this is a failed test and they have to go 
back through the process to improve it? Do they work 
together and value the input of others? These questions help 
to focus adults as well, to keep the purposes of engineering 
design as the important elements. 

Science Inquiry 

The inquiry portion of this lesson focuses on 
understanding attributes of different materials, and what 
attributes are key to making them float. After reading the 
story to the class the teacher asks, “What could have been 
in Oscar’s costume to make him float?” She asks questions 
to challenge student thinking about what attributes are 
important to make the material float (e.g., color is not an 
attribute that contributes to it floating). She shows the class 
a bag of sample materials and explains how they are going 
to test materials. She draws an “H” chart on the board (a 
chart with three columns). In the first column, she writes 
above the column “sinks,” and “gets soggy” over the 
middle column and “floats” over the final column. Students 
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are asked to draw this chart into their engineering journals 
before starting the tests. Each team gets a samples baggie 
(with small bubble wrap, foam, Styrofoam peanuts, 
cardboard, pompon, paper, newspaper, plastic from a 
grocery bag, paper clip, etc.) and a “tub” of water with 
about 3 inches of water in it (a gallon jug with the top cut 
off is a perfect size). 

Students test the materials and organize their data in the 
chart. When groups are done, a class discussion yields a 
class chart listing all of the items from the baggies. They 
then look closely at the items. that floated. What do they 
have in common? What can we say about all of them? This 
can end the session for the day. 

The Engineering Design Challenge 

The next lesson introduces the design challenge. A quick 
review of the story shows that Oscar’s costume floated, 
allowing him to rescue his friends. Reviewing the class chart 
from the previous investigation, the teacher can present the 
design challenge. Each team gets a 1'5” to 2” chunk of carrot 
with toothpick legs to represent “costumed” Oscar (Figure 
2). Again in their teams, students need to plan a costume that 
will float so that he can save his friends, using the materials 
they tested. Teams will need to draw a model of their plan, 
and make a list of materials they will need (they may have 
unlimited string to attach materials to Oscar since tape will 
not work well in water). In this plan, they need to decide 
why different materials are included; how will the materials 
work together as a system to make Oscar float? 

When they have created the prototype (Figure 2), they can 
test it in the water tub. If it does not float, teams can go back 
to the imagine stage and revise their plan and prototype. 
Even if it does float, they can work to make it even better. 

When teachers create the design challenge, it needs to be 
purposeful, even if the purpose is imaginative, and the 
materials to be used or other constraints are included in the 
directions. For example, students may “Use cereal boxes to 
create a flying machine for Leo to get back to the Waddatoy 
Toy Company,” a potential challenge for the book Leo, 
Cockroach. . .Toy Tester (O’Malley, 1999). Or, they might 
“Use toilet paper rolls, string, tissue paper, and rubber 
bands to create a musical instrument to send a message to 
someone you care about,” a potential challenge for the 
book Jove flute (Goble, 1992). Constraints are limits put on 
the design, for instance, the materials may be only those 
specified, or inventors have a specified amount of time. Still 
others might constrain some aspect of how students create 
the work, like team members may only use left hands, or 
maybe they cannot talk. For this challenge, the task itself 
was demanding enough for our second-graders that we did 
not add other constraints. 
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Sharing Strategy 

Once all teams (or most) have been successful, which may 
take two lesson periods, students can do a “museum walk” 
and “tour” the other teams’ work to look for similarities and 
differences. A class discussion, highlighting patterns of 
successful prototypes and how teams made materials work 
together in their designs wraps up the module. 


Misconceptions About Phases of the EDP 

As “simple” as the EDP looks, children do develop 
misconceptions about the steps and the model itself. It is 
helpful for teachers to be aware of the misconceptions in 
order to guide discussions and lessons in general to 
minimize misunderstandings. Understanding misconceptions 
can help teachers feel more confident because this 
knowledge can give teachers an idea of what kinds of wrong 
turns might occur as they try new lessons with their children. 
There is not one right way to solve a design challenge, and 
in fact there can be as many different solutions as children 
bring fresh ideas to challenges. 

After doing this series of lessons with K—2-aged children 
multiple times, misconceptions about the EDP have 
emerged. These recurring examples were collected through 
years of experience and not through a scientifically rigorous 
process. However, they can still be helpful for teachers in 
guiding elementary-aged children to develop habits of mind 
to facilitate problem solving and systems thinking. 

Problems can emerge at the onset with the ASK element. 
Children can have a difficult time identifying researchable 
questions. Kindergarten through second-grade students are 
also learning to identify declarative and _ interrogative 
sentences (National Governors Association Center for Best 
Practices & Council of Chief State School Officers, 2010), so 
when asked to listen for a question that needs a solution, as in 
this story “How can Oscar save his friends?” they identify 
any question in the story. So, when Oscar’s mom presented 
him with a hot dog costume, and the narrator asks, “And 
guess who was supposed to fit in the middle?” (Pilkey, 1995, 
p.10) second-graders hands wave in the air! They identified 
the sentence as a question, correctly for English Language 
Arts, but not a question requiring an engineering response. 
Even children who had attended a STEM school and talked 
about the EDP throughout first grade identified questions 
between characters as questions for the ASK part of the EDP. 

The IMAGINE step in the EDP is for brainstorming 
multiple ways to solve the problem (eie.org). In this case, 
multiple strategies for addressing the carrot to make it float. 
Seven- and eight-year-olds generally jump to the drawing, 
making one drawing and not thinking beyond that one 
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option. Encourage students to draw two or more different 
ideas, and then pick the best idea. 

This lesson does try to focus particularly on the making 
of the design sketches to show their thinking before they 
are able to get materials to create. What is interesting about 
initial plans, particularly with children who have not had 
much experience with the EDP is that they will choose 
materials that are unlikely to work, despite the prior 
experience with testing. Children who have not had many 
experiences with the EDP consistently do not utilize 
information from the previous investigation—even if the 
investigation just happened. Although the children from the 
STEM school were still having trouble identifying 
questions, they were often able to make the carrot float on 
the first try because they utilized materials from the “floats” 
list. Children who are new to this type of thinking 
sometimes need to be prompted with a question such as, 
“What kind of information on this chart might help you?” 

Last are the CREATE and IMPROVE steps. The idea of 
the lesson is to create their prototype, test it to see if it 
floats, and then if it does not, see if they can improve their 
design and try again. In an adult way of thinking this is a 
circular process, representing a failed trial, and a revision of 
the original design, leading to a new trial. In multiple times 
that this lesson was used, we hoped to document this 
process by having students call one of us over to take a 
picture or video record the trial, then come back and see the 
new trial, repeating for each trial until they were able to get 
it to float. Consistently with children new to the EDP, they 
do not call us over when they do test one. For the first 
several times doing this I thought they were not clear on the 
directions or that they got excited and forgot. I have worked 
to give the directions closer to the tnals, made 
announcements, and had more adults ready to record. Yet 
there is a bigger issue: children see this process as linear 
instead of circular. Instead of being able to go back and 
repeat steps, they see the task as a linear function: make the 
carrot float. In watching one particular class, when I was 
able to catch a group try their first prototype, I realized that 
after they put it in and it did not float, it got snatched out of 
the water and the group’s (unofficial) leader immediately 
went about making changes. But there was not a 
discussion, a review of their plan, or even a new plan made 
or discussed. It was not until they got it to float, that an 
adult would be called over. The task was completed. 


Conclusions 
Science teacher efficacy has been demonstrated to be a 
key index for measuring science teaching practice as those 
teachers who feel empowered by knowing both 
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methodology and science content tend to teach inquiry- 
based science in classrooms (Liang & Richardson, 2009). 
Thus, both knowing content and feeling confident and 
motivated to teach hands-on science with designs that call 
for students conducting experiments is dependent upon a 
teachers’ self-beliefs about their science knowledge. Lack 
of science efficacy leads to teacher avoidance of inquiry- 
based science. As prevalent as this problem is with 
elementary teacher, the lack of teachers’ engineering 
efficacy is even greater. 

In order to support teachers and students in inquiry-based 
science advocated in the current article, we suggest a review 
of Mott, Chessin, Sumrall, Rutherford, and Moore (2011) as 
the assessment of students’ science products generated in 
inquiry-based science experiences can be supported with 
rubrics that align teacher-student and _ student-student 
discussions with science objectives. See the rubric online at: 
http://ojs.jstem.org/index.php?journal= JSTEM& page= index. 

EDP “misconceptions” are important to understand. 
Students come to school with cultural and age-related 
conceptions about processes (Bransford, Brown, & 
Cocking, 1999). Teachers need to grasp the fact that this is 
an underpinning of understanding that teachers should not 
find fault for but realize that children need exposure to these 
alternate models of thinking. Particularly as we prepare 
more children to be college and career ready, children will 
only learn to persevere, for instance, if they understand that 
they have failed many times in their lives, and that it is not a 
bad thing, but you keep going and trying new strategies 
until you reach success. Or that the means to an end is 
multiple steps along the way, and that the end may not 
necessarily be the end, it may just start another question, 
such as How can we create lessons to dispel misconceptions 
about using the EDP? Teachers can find the EDP a useful 
thinking process that weaves throughout learning. 
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Prior research regarding minorities in the science, technology, engineering, and mathematics (STEM) fields indicated 
that the factors of peer support and participation in STEM-related activities contributed positively to minority students’ 
recruitment and retention in these fields. Utilizing stereotype threat as a conceptual framework, this qualitative case study 
investigated the contribution of these factors by examining the experiences of minority students majoring in a STEM field. 
Data analyzed through a deductive approach indicated that a stereotype threat adversely affecting retention and 
recruitment in STEM could be either minimized or maximized by peer support, in the form of educator or familial 
influence, and participation in STEM-related activities through internships and the application of STEM-related 
knowledge. Findings from this research may inform higher education institutional practices involving the recruitment and 


retention of students to the STEM fields from minority populations. 


There is a need to increase the number of minority 
students who major in a science, technology, engineering, 
or mathematics (STEM) fields. Historically, white males 
have dominated the population of professionals within 
these fields. Diversification of individuals within STEM 
careers can contribute new insights, viewpoints, and 
interpretations that aid the advancement of those fields in 
various ways. Therefore, increasing the strength of the 
various sections of the STEM pipeline (e.g., recruitment 
and retention) in higher education should be a goal. The 
STEM fields have historically seen little participation from 
minorities, although the numbers from this group are 
slowly increasing (Huang, Taddeuse, Walter, Samuel, & 
Washington, 2000, p. 3) and institutions of higher 
education have made great strides in attempts to recruit and 
retain STEM minority majors. In addition, STEM industries 
have become the fastest-growing sections of the economy 
and job growth (Vilorio, 2014). As the population of the 
United States continues to increase and become more 
racially and ethnically diverse, the demand to fill those jobs 
can be partially satisfied through an increase of the number 
of minorities entering the STEM fields. The United States 
Department of Education (2016) predicted that in the years 
following 2014, the minority student population in public 
elementary and secondary schools would become the new 
majority. Therefore, recruiting and retention efforts are 
imperative to ensure that more minority students pursue 
opportunities in STEM fields in K-16 and that their 
numbers increase at all levels of education. 

Research has been conducted in order to determine those 
factors that contribute to an individual’s initial decision to 
major in a STEM field, as well as persist in the field 
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(Beasley & Fischer, 2012; Cole & Espinoza, 2008; Griffith, 
2010). However, much more information is needed in order 
to close the gaps in the number of STEM professionals 
from minority populations. Great insight can be directly 
gained from minority students who have currently chosen a 
major in higher education in one of these fields. Through 
the interpretations of student experiences, knowledge can 
be expanded in order to formulate retention strategies for 
increasing the number of minorities that persist in the 
STEM fields. Still unknown are definitive factors that 
contribute to persistence in STEM fields; therefore, no 
focus has been placed on particular strategies that can 
improve this persistence (Griffith, 2010). This research 
project sought to uncover those features that contributed to 
minority student’s perseverance, retention, in the STEM 
field based on the experiences of current minority STEM 
majors. Specifically, this study sought to answer the 
following questions: Why do minority students choose to 
pursue a degree in a STEM field? and What factors 
contribute to minority student’s persistence for studying in 
their respective field? 

Implications for practice from this study comprise the 
development of recruitment strategies to increase the 
number of minority students who pursue a degree in a 
STEM field. If there are specific experiences of minority 
students that allow for increased engagement in the STEM 
fields, then replicating such experiences would be desired. 
Researchers have noted far fewer minority students 
pursuing bachelor’s degrees in general (Griffith, 2010), 
which has affected the number of minority students 
receiving bachelor’s degrees in a STEM field. Through 
interviewing minority STEM students, this study aimed to 
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uncover and document those experiences that contributed 
to students’ engagement and persistence in a STEM field. 


Terminology 

Ever since Judith Ramaley, former director of the 
National Science Foundation’s Education and Human 
Resources Division, defined STEM education (Teaching 
Institute for Excellence in STEM, 2010), various 
professional organizations, institutions, and researchers have 
voiced opposing opinions concerning clear categorizations 
of the specific disciplines that compose STEM (Koonce, 
Zhou, & Anderson, 2011). For purposes of this study, the 
disciplines classified as STEM will be those described by 
Koonce et al. (2011) as “high-frequency STEM education 
fields” (p. 8), or the general fields of mathematics, physics, 
chemistry, biology, computer science, and engineering. 

The definition of the term minority utilized in this study 
corresponds to those individuals who identify as Asian 
American or Pacific Islander, African American, Hispanic, 
or Native American. In concert with prior research by 
Museus, Palmer, Davis, and Maramba (2011), this study 
has not included mixed race individuals as defined by 
governmental agencies (U.S. Census Bureau, 2004a, 
2004b). Furthermore, this particular research study utilized 
a general approach to minority membership in order to 
allow some freedom in the representation of the data as a 
wide-ranging study on minority STEM majors. 


Literature Review 

Prior research regarding minority students have sought to 
uncover those factors that contribute to persistence and 
retention, and research specific to minority STEM majors 
has pursued similar themes (Chang, Sharkness, Hurtado, 
& Newman, 2014; Cole & Espinoza, 2008; MacPhee, 
Farro, & Canetto, 2013). While very few factors have 
been singled out as contributing to the retention or 
attrition of minority students pursing STEM majors, 
agreement has been found that more study and 
evaluation is merited. Griffith (2010) maintained that 
increasing the number of students who choose to pursue 
a degree in a STEM field should be a priority. The 
reduced number of minorities in STEM fields have been 
identified as a continuing critical problem in the United 
States (Beasley & Fischer, 2012). The Higher Education 
Research Institute (2010) found that of those students 
from a minority background that did chose to major in a 
STEM field, only 16% finished their degree within five 
years. Therefore, determining what factors contribute to 
students’ interest in STEM has and will continue to be of 
great concern. 
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A Case for Qualitative Methods 

Most of the research involving recruitment and retention 
of minority students in STEM education has employed 
quantitative techniques (Chang et al, 2014; Cole & 
Espinoza, 2008; Hurtado et al., 2007; MacPhee et al., 
2013). Studies involving qualitative research methods with 
regard to minorities in STEM education are necessary in 
order to provide information not found in quantitative 
studies. In fact, Museus et al. (2011), in their research on 
the current literature regarding those factors that contribute 
to minority STEM success, conveyed “the importance of 
moving beyond simplistic aggregated analyses to more 
complex understandings of the experiences and outcomes 
for various ethnic groups” (p. 20). Erlandson, Harris, 
Skipper, and Allen (1993) make a case for qualitative 
research by creatively portraying the rich expressive data 
that can result from naturalistic inquiry. 

Relatively few qualitative research methods studies have 
been conducted to probe those factors that contribute to 
recruitment and retention of minority students in STEM 
(Palmer, Maramba, & Elon, 2011). Additionally, during the 
review of the literature, only one mixed methods study was 
found that involved qualitative research methods and all 
categories of minority STEM majors (Buschor, Berweger, 
Frei, & Kappler, 2014). Buschor et al. (2014) studied 
stereotype threat as a factor of STEM success. The authors 
included women in the study, another underrepresented 
STEM group. 

Due to the limited amounts of qualitative research 
involving minority STEM majors, the study conducted by 
Palmer et al. (2011) served as the primary foundation for 
this study. This was predominantly due to these researchers 
utilization of the qualitative research method of interviews 
to determine those factors that contributed to the 
recruitment and persistence of minority STEM majors. 
Palmer et al. (2011) noted three themes that contributed to 
continuation and persistence in the STEM field. Peer 
support, participation in STEM-related activities, and 
strong secondary and elementary preparation influenced 
minority students’ success in completion of a STEM field 
degree (Palmer et al., 2011). Results from the present study 
may corroborate these themes or uncover other various 
factors that may impact recruitment and persistence of 
minority STEM majors. 

Recruitment and Retention Factors 

The review of the literature also indicated that self- 
concept and self-efficacy affected the success of a minority 
student in the STEM fields (MacPhee et al., 2013), 
however, the volume of research in those areas was 
somewhat limited (Leslie, McClure, & Oaxaca, 2014). 
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MacPhee et al. (2013) found that students who persisted in 
a STEM field until graduation had higher levels of 
academic self efficacy, provided that these students also 
participated in a mentorship program. MacPhee et al.’s 
(2013) research asserted that retention of minority students 
in a STEM field was maintained through the use of 
mentorship programs. 

Other researchers have quantitatively determined 
additional factors that contributed to the recruitment and 
retention of minority STEM majors. For example, Cole and 
Espinoza (2008) analyzed college grade point averages 
(GPA’s) for minority students studying in a STEM field 
and whether or not particular environmental variables such 
as attending diversity related functions, student-faculty 
interactions, and negative feedback about academic work 
significantly contributed to GPA. Results from Cole and 
Espinoza’s (2008) study designated that attending diversity 
functions was statistically significant for lowering a 
minority student’s GPA. This finding contradicted previous 
research suggesting that attending events on campus that 
promote diversity would positively impact educational 
endeavors (Montelongo, 2003). 

In addition, Hurtado, Carter, and Spuler (1996) 
documented that familial support and maintaining familial 
bonds were valuable aspects to transition and successful 
completion of a college degree. Gloria, Castellanos, and 
Rosales (2005) found that a perceived positive university 
environment, increased cultural congruity, and a decrease 
in the number of perceived barriers were related to 
persistence in higher education for minority Hispanic 
students. To summarize, similar research conducted with 
regard to minority STEM students found that the 
environment of the University and the respective STEM 
departments can have a great effect on the decision of a 
minority student to major in the STEM field (Griffith, 
2010). - 


Conceptual Framework 

Stereotype threat has been identified as one of the factors 
that may contribute to the retention and recruitment of 
minority students in a STEM field (Museus et al., 2011). 
Stereotype threat is the self-perceived concept and the 
apprehension caused by the idea that engaging in certain 
behaviors may confirm negative attributes commonly 
associated with minority group membership (Steele, 1997; 
Steele & Aronson, 1995). Stereotype threat was first 
described by Steele and Aronson (1995) in their study of 
how this socio-psychological notion affected the 
intellectual performance of African Americans. Steele and 
Aronson assigned African American and White students of 
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similar intellectual abilities to two different groups. For one 
group a stereotype threat was introduced while the other 
served as a control. Each group was given a diagnostic 
exam of intellectual ability. Steele and Aronson found that 
academic performance of the African American students 
was significantly lower than their White counterparts when 
a stereotype threat was perceived. Since the inception of 
stereotype threat, other research has demonstrated the 
applicability of this idea to other groups often associated 
with a stereotype, such as females in mathematics (Inzlicht 
& Ben-Zeev, 2000) and students from low socioeconomic 
status (Croizet & Claire, 1998). 

Stereotype threat was also known to be present when a 
particular minority group experiences the pressure to 
“disconfirm” the pertinent stereotype (Fogliati & Bussey, 
2013). Furthermore, stereotype threat can describe the 
deficit in performance over several domains among 
minority groups (Beasley & Fischer, 2012). Results of 
these studies conveyed that cognizance of the stereotype 
was adequate enough to reduce minority intellectual 
performance (Steele & Aronson, 1995). There have been 
conflicting reports expressing that stereotype threat can 
cause motivational pressure, which can increase effort and 
success (Fogliati & Bussey, 2013; Jamieson & Harkins, 
2007). For this study, stereotype threat will be used as a 
guide to understand what, if any, contributions this concept 
has on recruitment and retention of minority STEM majors. 


Methodology 

Lincoln and Guba (1985) noted, “objectivity exists when 
an appropriate methodology is employed” (p. 300). In order 
that objectivity be maintained, the qualitative research form 
of this investigation followed a case study design. Case 
studies encompass a complete description and an analysis 
of a particular instance contained within a certain context or 
boundary (Merriam, 1998). The instance of this study 
involved the experiences of minority STEM majors, while 
the context of the study occurs at a regional university in 
the Southwest. Case study emphasizes the complexities of a 
particular individual or program of special interest (Stake, 
1995). While this study does not involve a specific program 
per se, the STEM fields and the issue of increasing the 
number of minorities within the fields lends itself to 
consideration as a program of special interest. Likewise, 
Merriam (1998) ascertained that a case study was the 
appropriate qualitative form when development of an 
understanding of the changing aspects of an individual or 
program was warranted. 
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Table 1 
Classroom Observations 


= ess SS 





Number of Instructor’s Field Course Subject Course Classification Level 
Observations 

3 Mechanical Engineering Mechanical Engineering Senior 

3 Engineering Technology General Engineering Freshmen 

3 Mathematics General Calculus I Sophomore 

3 Mathematics General Calculus II Sophomore 

3 Integrated Mathematics and Physics General Physics Freshmen 

Participants provided “access to the most complicated social and 


There were 57 undergraduate minority students who 
participated in this study. The students were studying in 
high-frequency STEM fields and from all classifications 
(freshmen, sophomore, junior, or senior); 50 answered 
surveys and 7 were interviewed. All participants were 
currently enrolled in an institution of higher education. A 
database of minority STEM students was generated at the 
institution and a recruiting email was sent to the students. 
Data Sources 

Observations, interviews, and document artifacts served 
as sources of data and assisted in the effort to triangulate the 
data for effectiveness and trustworthiness. Data collection 
began with observations of various higher education STEM 
classrooms and collection of document artifacts in the form 
of survey responses and seating charts. Semi-structure 
interviews were guided by themes and ideas that emerged 
from the observations and document artifacts. 

Nonparticipant observations of minority students in a 
respective STEM major classroom were conducted 
throughout the course of a long semester. Observations 
documented the minority student’s experience in the 
respective major course with emphasis on interactions with 
the professor and peers. Spradley (1979) maintained that 
selecting a social situation to conduct research should 
follow the selected criteria of simplicity, accessibility, 
frequently recurring activities, and participation. Table 1 
depicts the various subjects and classifications for the 
classrooms in which observations occurred. Surveys as 
document artifacts were chosen due the fact that “artifacts 
can be technological devices ..., works of art, writing 
instruments, tools, and almost any other physical evidence” 
(Erlandson et al., 1993, p. 100). According to Merriam 
(1998) surveys are considered as researcher-generated 
artifacts and the “specific purpose for generating documents 
is to learn more about the situation, person, or event being 
investigated” (p. 119). Fifty students responded to the 
survey questions. 

The final data source for this study consisted of 
interviews with minority STEM students. Interviews 
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educational issues...abstractions based on concrete 
experiences of people” (Seidman, 2006, p. 1). Gubrium and 
Holstein (2003) posited that interviews allow for empirical 
data collection in a social aspect so that individuals may 
talk freely about their experiences and their lives. Seven 
interviews were conducted as part of this study, however 
interview data from only three students has been presented 
as part of the findings due to manuscript length constraints. 
These students (pseudonyms assigned) include Jose, a 
Hispanic senior computer science major; Anita, a Hispanic 
freshmen biology major; and Maria, a Hispanic sophomore 
biology major. 

Data Analysis 

Data analysis techniques for the interview and 
observation data consisted of color-coding techniques in 
order to develop categories and disaggregate core 
groupings that were relevant in providing answers to the 
research questions. Consistent with the deductive approach 
of this study, and based on prior research from Palmer et al. 
(2011), overarching themes of peer support and 
participation in STEM-related activities were first color 
coded. Each data source was color coded individually for 
these general categories. 

Following this initial categorization, several iterations of 
coding were conducted using only the data that conformed 
to the general categories of peer support and participation in 
STEM-related activities. This allowed for saturation of the 
original categories and to allow new categories or 
subcategories to emerge (Erlandson et al., 1993, pp. 118- 
119). At this point, the new subcategories were compared 
between the different data sources in order to address the 
common categories established within the interviews, 
surveys, and observation notes. Subcategories were only 
included if they were present among all the three sources of 
data collection. 

Trustworthiness 

Trustworthiness was evaluated based on credibility, 
transferability, dependability, and confirmability. While 
every effort was made to ensure the trustworthiness of this 
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study Seidman (2006) argued that formulaic approaches 
were not necessary so long as respect and empathy for the 
issues of trustworthiness were maintained. Credibility was 
maintained through the use of member checks with 
interview participants and the interpretations formed as 
the interview unfolded. Every effort was made to assess 
the data for credibility and truthfulness, however 
Rossman and Rallis (2003) maintained that in qualitative 
studies, the data must be approached with humility and 
must be useful in answering the research questions while 
understanding that the data is in the perspective of the 
participant, not based on the researcher’s notions. 
Credibility was also maintained through the use of an 
audit trail which consisted of line numbering in the 
interview transcriptions, observation notes, and surveys 
from which direct quotes can be traced. 

Transferability can be extended as a qualitative research 
methodology that supports existing research. The data from 
this study also supported similar themes developed by Palmer 
et al. (2011), which further enhanced trustworthiness through 
transferability. Lincoln and Guba (1985) recommended that 
“anyone seeking to make a transfer is to accumulate empirical 
evidence about contextual similarity; the responsibility of the 
original investigator ends in providing sufficient descriptive 
data to make such similarity judgments possible” (p. 298). 
Reliability was achieved through the repeated means of data 
collection. Lincoln and Guba stated that reliability is achieved 
through demonstrated use of replication and inquiry within 
similar conditions. Triangulation through the use of document 
artifacts, interviews, and observations has contributed to the 
confirmability of this study. 


Limitations 

During the recruitment of faculty that provided 
opportunities for classroom observations for this study, 
communication was first-initiated with the chairs of the 
departments of the mathematics, chemistry, and physics 
department; engineering and computer science department; 
and the life, earth, and environmental science department. 
Responses were received from the department chairs from 
the mathematics, chemistry, and physics and engineering 
and computer science department and as a result, faculty 
were recruited. No response was received from the life, 
earth, and environmental science department, although 
multiple methods of communication were attempted. This 
was considered a limitation of this study, as three of the 
seven interviews conducted originate from students in the 
biological sciences. 

Time also served as a limitation of this study in two 
ways. The first being the time frame for collecting data 
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through observations, interviews, and surveys. Permission 
for recruitment of participants for this study was not granted 
until mid-term of the long semester and, therefore, hastened 
the process of data collection to be completed before 
students completed final exams. Second, there was 
difficulty in coordinating observations of STEM classrooms 
due to teaching schedules of the faculty. 

Although adding credibility and trustworthiness to the 
study, the researcher acknowledges personal experience as 
a STEM major from a minority background may be a 
possible limitation of this study. Personal experiences 
served as a primary catalyst for beginning this research 
study yet added a dimension of researcher bias, for the 
researcher approached the study with knowledge and 
personal experience of being a minority STEM major. To 
mitigate this particular bias, a deductive approach was 
utilized by considering the categories developed by the 
research conducted by Palmer et al. (2011). In this light, it 
must be noted that findings were developed based on prior 
research findings and not the researcher’s bias. 


Findings 

Consistent with prior research findings from Palmer et al. 
(2011), the categories of peer support and participation in 
STEM-related activities were found to be present in all 
sources of data collection. Based on color-coding strategies 
in data analysis, these general categories were then broken 
into subcategories. Subcategories found to be present under 
the overarching category of peer support include educator 
and family influence. Subcategories present under the 
category of participation in STEM-related activities 
comprised internship experience and application of 
knowledge in a STEM field. All of the data, which fell into 
these subcategories, were then divided into whether or not 
the piece fell under the recruitment or retention of minority 
students within the STEM field. Table 2 depicts the major 
categories and subcategories developed within this study. 
The presentation of findings begins with the categories of 
peer support for recruitment, peer support for retention, 
participation in STEM-related activities as a recruitment 
tool, and finally participation in STEM-related activities for 
retention. For audit trail purposes, survey respondents were 
coded. For example, in the code S1Q1, the S1 indicates the 
survey respondent number and Q1 indicates the question 
responded to within the survey. 
Peer Support for Recruitment 

A subcategory most frequently present throughout the 
data was that of educator influence in recruitment of 
minority students within the STEM field. Using the surveys 
as document artifacts, 14 responses within the survey 
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Table 2 
Categories and Subcategories of Recruitment and Retention 


Recruitment 


Peer Support 
Educator Influence 
Family Influence 

Participation in STEM-Related Activities 
Application of STEM Knowledge 
Internship Experience 


Retention 


Peer Support 
Educator Influence 
Peer Influence 

Participation in STEM-Related Activities 
Application of STEM Knowledge 
Internship Experience 





specifically referred to a previous high school teacher as 
being a major inspiration or role model for pursuit of a 
STEM degree. Within the interviews, the notion of 
educator influence on the recruitment of minorities in 
STEM was also present. Anita mentioned a high school 
teacher conveying that this teacher “was really, she was 
really good. And really fun. And she never took shortcuts, 
she always showed us the long way but it was worth it” 
(Anita, pp. L184—185). 

Findings of specific interest included three survey 
responses that referred to a high school math teacher as 
having an influential factor in the decision to major in a 
STEM field. This sentiment was echoed by Jose, a 
computer science major, who shared 


While I was in high school I was mostly leaning 
towards psychology and this math teacher of mine kept 
telling me that not to go for psychology, that I was good 
in math and that I should just go for something that had 
a lot of math. (Jose, pp. L226—229) 


While much of the data indicated support through 
educator influence in a positive light, a senior Hispanic 
female biology major survey respondent indicated that 
although for her the educator influence was negative, it 
motivated her more to achieve a degree in her respective 
STEM field. She described the experience by stating that “T 
never passed a science class in high school... then got told 
I would be nothing in life by my advisor so I chose to prove 
them wrong and that’s what inspired me” (S40Q10). 

Family influence also contributed to recruitment of 
minorities within a STEM field course of study. When 
asked to describe what inspired the student to pursue a 
major in a STEM field, a junior Asian American female 
biology major responded to the survey question by stating 
that her mother served as inspiration “considering she was 
the only one to finish with a college degree when they came 
to America” (S16Q5). Other role models for minority 
STEM majors included family members who have also 
successfully pursued degrees in a STEM field. During the 
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interview with Jose, he indicated a familial influence for 
recruitment to major in computer science. Jose stated that 
“an uncle of mine does software programming for like, um, 
libraries and stuff like that and he goes everywhere with 
that” (Jose, pp. L231—232). 


Peer Support for Retention 
Educator influence was also a frequently present factor 
contributing to the retention of minorities within a STEM 
field. A senior Hispanic female biology survey respondent, 
replied im the following manner to describe the best role 
model for completion of her STEM degree: 


Dr. Gershen (pseudonym)! He is an awesome teacher. 
He has taught me everything I know and has pushed me 
to realize what I really wanted to do in life but thought I 
was too dumb to do it. He completely inspires me and I 
could not thank him enough! (S28Q10) 


Another survey respondent, a junior Native American 
male chemistry major, stated that when he needed assistance 
in his STEM courses, the professor’s help was critical. “He 
has instructed me through my majors courses and I know 
that I would not have the accomplishments I do today 
without their guidance. I will be forever grateful” (S12Q9). 

Educator influence as it related to retention was detected 
in the STEM major classroom observations. In an 
observation of a calculus I course, the interaction between a 
Hispanic male mathematics major and the instructor 
showed high engagement in the material by both parties. 
When debriefing the instructor, he commented that this 
student was always highly involved during the lecture and 
did well, even though the student himself had the idea that 
he was not doing well. 

Peer influence in the form of tutors, classmates, and 
friends was another key category that appeared in the 
analysis of the data to support or produce barriers to STEM 
degree completion for minority students. In 23 out of the 50 
surveys, respondents indicated that when assistance was 
needed in their respective STEM major courses, they often 
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sought out the help of a tutor or friends. One survey 
respondent, a freshman Hispanic female mechanical 
engineer, verbalized that one barrier to STEM degree 
completion was “being the only female in some classes, 
having people tell me I should switch majors when I slip 
up” (S14Q7). This same respondent went on to state the 
most frustrating thing about majoring in a STEM field at 
this university was “people who have this underlying 
assumption that im [sic] not as smart as my white male 
classmates” (S14Q8). This type of comment was reiterated 
in the survey responses from a sophomore Hispanic female 
mechanical engineering major who stated there were “Not 
many femals [sic] even less that are Hispanic” (S9Q), 
perceiving this was a barrier to her earning a STEM degree. 
An interesting finding emerged from Maria’s interview. 
She referred to a sense of community that evolved with 
other minority peers in her course of study. In order to 
receive assistance with the course material, Maria attended 
an organized study group led by a student paid through a 
university service. Until she was asked by the researcher to 
describe the other minority students majoring in biology, 
Maria was unaware that all of the students that attended the 
study session were minorities, including the student leader. 
Peer support for retention was apparent in the observations 
of the STEM classrooms and was detected through minority 
students tending to group together, which occurred in all the 
classroom observations. This was especially evident in the 
seating choice of minorities attending the STEM classes. All 
of the STEM classrooms were organized into rows and 
columns for seating, with students having a choice of seat 
placement. When considering a 3 by 3 square of seating with 
a minority student seated in the middle seat of the middle 
row, 76% of such seating arrangements saw three or more 
minority students seated within the 3 by 3 square, in addition 
to the one seated in the middle. 
STEM-Related Activities and Recruitment 
Survey respondents indicated that STEM-related activities, 
such as starting a school robotics team, involvement in a 
science fair, membership in a welding team, and participating 
in an engineering club, contributed to the minority students’ 
interest in continuation of study in a STEM field at the 
postsecondary level. One survey respondent, a freshman 
Hispanic female biology major, asserted the importance of 
STEM-telated activities for recruitment by stating that “it was 
such an exhilarating experience to dissect animals. Apart 
from my awesome teacher, I also enjoyed our field trip to a 
medical school and seeing human cadavers” (S17Q6). 
Internships as a form of participation in STEM-related 
activities also served as a recruiting tool for minority STEM 
majors. One survey respondent commented that an 
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internship with a family health clinic inspired her to pursue 
a major in biology, as did another survey respondent who 
participated in an internship through the high school she 
attended. Anita referenced an internship experience in 
which she aided missionaries in providing medical care to 
individuals in a third world country. For Anita, the 
experience was the most influential factor in her life that 
affected her pursuit of a major in biology. 
STEM-Related Activities and Retention 

Application of knowledge in a STEM field through 
STEM-telated activities was also categorized as a retention 
factor for minority STEM students at this university and 
served as a catalyst for majoring in a STEM field. For 
example, an African American computer science major 
referenced his love of technology as inspiration for 
choosing his major and stated that “I started thinking if I 
like engineering so much I should engineer something that I 
love, which computers was that something” (S21Q5). 

Observation of the freshmen level engineering and the 
sophomore level calculus II class also confirmed this notion 
of application of knowledge as important to the retention of 
minority STEM students. Specifically, in these two classes 
the minority students were more engaged and involved in 
the particular STEM content as they were applying their 
knowledge through STEM-based activities. In the freshman- 
level engineering course, students were drawing and 
designing various objects using a compass, protractor, and 
straight edge. In the sophomore calculus II class, students 
were engaged in working problems that were real-world 
application based. 


Discussion 

Any discussion of the findings must begin with revisiting 
the conceptual framework’s relationship to the categorical 
findings and the recruitment and retention of minority 
STEM majors. In reference to educator and peer support for 
recruitment, a stereotype threat adversely affecting 
recruitment in STEM could possibly be minimized or 
maximized through the influence of an educator or family 
member. This was apparent in the decision for Jose to major 
in computer science after having a teacher comment on his 
excellence in mathematics and despite his reluctance to 
major in a STEM field because of his self-doubt of being 
able to excel. Confirmation of this finding existed in the 
survey comments about the robotics teacher who pushed 
one student to “pursue those things which I was afraid of.” 
These findings are consistent with prior research conducted 
by Rivardo, Rhodes, and Klein (2008), who found that 
stereotype threat was less likely to affect academic 
performance with a cultivating and personable environment. 
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Aronson et al. (1999) asserted that stereotype threat 
interference with performance does not require a history of 
negativity or feelings internalized relating to academic 
inferiority, but can solely originate from situational 
pressures. 

Sackett, Hardison, and Cullen (2004) referred to the 
hypothesis that stereotype threat in a situation becomes 
striking when concerns about being judged with regard to a 
stereotype arise to adversely affect performance. Familial 
influence as a recruitment tool for minority STEM majors 
allowed for support in the form of a mentor, someone who 
has already achieved a STEM degree. This finding was 
evidenced throughout the surveys and interviews and also 
confirmed the findings of MacPhee et al. (2013) who 
uncovered that mentorship positively affected retention. 
Likewise, Museus et al. (2011) indicated in their review of 
the literature that parental involvement and support can 
facilitate the recruitment of minority students within a 
STEM field. Knowing others who have completed a STEM 
degree with success may allow for those stereotype threat 
concerns to be voiced and addressed by someone who may 
have had the same experiences. Good, Aronson, and 
Inzlicht’s (2003) study determined that effects of stereotype 
threat can be reduced or eliminated when psychological 
interventions were applied. 

Carr and Steele (2009) maintained that the burden of 
negative stereotypes about one’s group interferes with 
one’s capacity to adapt to new situations. Inability to adapt 
causes individuals to revert to attributes, attitudes, and 
situations familiar to them. As evidenced in all data 
sources, students within the STEM courses often conversed 
only with other minority students, studied with other 
minority students, and sat next to other minority students. 
Museus et al. (2011) suggested that minority students in 
general “feel increased susceptibility to being stereotyped, 
which could make them more vulnerable to stereotype 
threat” (p. 36). In the study conducted by Good et al. 
(2003), mentorship by college student peers was shown to 
minimize and in some instances eradicate stereotype 
threats. Minority students relying on themselves for peer 
support was evidenced and possibly provided a certain 
level of mentorship. 

Therefore, in response to the first research question, 
findings from this study indicated that educator and familial 
influence as well as participation in STEM-telated activities 
with respect to application of knowledge and internship 
experience affected recruitment of minority students to a 
STEM field. Within these categories and following from 
the data sources, educators have the most influence toward 
recruitment of minority STEM majors. 
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In regard to the second research question, factors that 
contributed to persistence and therefore retention of 
minority STEM majors were educator, classmate, peer, or 
tutor influence and participation in STEM activities 
involving the application of knowledge and internships. 
Strong factors for retention of minority students who pursue 
a STEM degree included the educator and peer influence. 

Suggestions for future studies include exploring the 
conceptions that mathematics or mathematics teachers have 
regarding minority STEM recruitment and retention. 
Findings from this study indicated that this was a common 
subcategory not fully explored within the data collected and 
should be examined further. Other research studies should 
investigate what factors mathematics in general has on 
minority STEM recruitment and retention. Museus et al. 
(2011) contended that the literature suggested that African 
Americans and Hispanics were placed into remedial 
mathematics courses at disproportionate rates, which would 
affect their success in many STEM fields. Implications for 
practice from this study reiterate the idea of the impact and 
influence an educator has on the life of students, more 
specifically minority STEM students. Educators in the 
STEM fields must be made aware of their impact on 
students from these populations and efforts be made to 
inspire their decision to major in STEM. Moreover, 
instructors in higher education must also be aware of the 
impression they make with retaining these students once a 
STEM course of study is chosen. 


Recommendations and Conclusion 

As stated previously in the review of the literature, more 
research is necessary in order to increase the number of 
minority students who pursue a STEM field. Stereotype 
threat has been known to adversely affect academic 
performance (Aronson et al., 1999; Beasley & Fischer, 
2012; Carr & Steele, 2009; Croizet & Claire, 1998; Fogliati 
& Bussey, 2013; Sackett et al., 2004; Steele, 1997; Steele & 
Aronson, 1995). However, Good et al. (2003) showed that 
this threat can be managed or eliminated as a factor affecting 
success in minority groups. Determining the factors that 
positively contribute to minority STEM major recruitment 
and retention, may allow for the elimination of stereotype 
threats that hinder academic success for minority students in 
the STEM fields. Increasing the numbers of minorities in 
STEM careers should be a goal of many institutions in order 
to diversify the number of professionals in these areas. 
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Ted Eisenberg, Section Editor 





This section of the Journal offers readers an opportunity to exchange interesting mathematical problems and solutions. 
Please send them to Ted Eisenberg, Department of Mathematics, Ben-Gurion University, Beer-Sheva, Israel or fax to: 
972-86-477-648. Questions concerning proposals and/or solutions can be sent e-mail to eisenbt@013.net. Solutions to 
previously stated problems can be seen at http://www.ssma.org/publications. 


Solutions to the problems stated in this issue should be posted before March 15, 2018 


¢ 5475: Proposed by Kenneth Korbin, New York, NY 
a+b=14V/ab—48, 

Given positive integers a, b, c, and d such that ¢ b+c=14V/bc—48, 
ct+d=14V/cd—48, 


with a < b<c< _d. Express the values of b, c, and d in terms of a. 


* 5476: Proposed by Ed Gray, Highland Beach, FL 
Find all triangles with integer area and perimeter that are numerically equal. 


* 5477: Proposed by Daniel Sitaru, “Theodor Costescu” National Economic College, Drobeta Turnu-Sevrin, Mehedinti, 
Romania 


Compute: 
— Lav 1 tx? V1 tx2- 00 Wx? 
L= lim { Inn+ ba ak aaa 
n—co x— X 


* 5478: Proposed by D. M. Batinetu-Giurgiu, “Matei Basarab” National Collge, Bucharest, Romania and Neculai 
Stanciu, “George Emil Palade” Secondary School, Buzau, Romania 


Compute: 
7/2 i 1g 1 1 
| cos x(sinx sin (Fos) +cosxsin?(=sinx) )dx, 

0 


* 5479: Proposed by José Luis Diaz-Barrero, Barcelona Tech, Barcelona, Spain 
Let f : [0,1] — # be a continuous convex function. Prove that 


9 1/3 3 2/3 8/15 
Ar f(ar+ | soa >>| f (t)dt. 


0 — 8 Jo 
* 5480: Proposed by Ovidiu Furdui, Technical University of Cluj-Napoca, Cluj-Napoca, Romania 
Let n > 1 be a nonnegative integer. Prove that in C[0, 27] 
span{1,sinx, sin (2x),...,sin (nx)}=span{1,sinx, sin*x,...,sin"x} 
if and only ifn = 1. 


k 
We mention that span{y, v2,.. Veh = > ajvj, 4; © R,j=1,...,k, denotes the set of all linear combinations with 
Vilas VDis, evel enn Vie j=l 
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This section of the Journal offers readers an opportunity to exchange interesting mathematical problems and solutions. 
Please send them to Ted Eisenberg, Department of Mathematics, Ben-Gurion University, Beer-Sheva, Israel or fax to: 
972-86-477-648. Questions concerning proposals and/or solutions can be sent e-mail to eisenbt@013.net. Solutions to 
previously stated problems can be seen at http://www.ssma.org/publications. 


Solutions to the problems stated in this issue should be posted before April 15, 2018 


¢ 5481: Proposed by Kenneth Korbin, New York, NY 
A triangle with integer area has integer length sides (3, x,x+1). Find five possible values of x with x > 4. 


* 3482: Proposed by Daniel Sitaru, “Theodor Costescu”’ National Economic College, Drobeta Turnu-Severin, Mehe- 
dinti, Romania 
Prove that if 7 is a natural number then 


(tan 5)” (tan4 )” (tan3 )” = 3 
(tan4 Me (tanBe) &@ (tan3-)e-r (tan) -2( tan 2-)—-— (fare ee eee 
* 5483: Proposed by D. M. Batinetu-Giurgiu, “Matei Basarab” National College, Bucharest and Neculai Stanciu, 
“George Emil Palade” School Buzau, Romania 


Ifa, b > 0, and x € (0,2) then show that 


sinx 2ab , tanx 6ab 
I. (a+b) eT Jae ¥ Bs: 


2. a: tanx+ 6: sinx > 2xVab. 


¢ 5484: Proposed by Mohsen Soltanifar, Dalla Lana School of Public Health, University of Toronto, Canada 

Let X;, X> be two continuous positive valued random variables on the real line with corresponding mean, median, and 
mode X;, X1, X; and Xz, X2, X2 respectively. Assume for their associated cumulative distribution functions (CDFs), we 
have 


Fy,(t) <Fuy(t) (> 0). 


Prove or give a counter example: 

(a) x2 < x1, (0) <%1, (©) %2 <%.° 

* 5485: Proposed by José Luis Diaz-Barrero, Barcelona Tech, Barcelona, Spain 
Let x, y, z be three positive real numbers. Show that 


[[ @x+3yt2+1) $5 (4x+2yt1)° > 3. 


cyclic cyclic 


* 5486: Proposed by Ovidiu Furdui, Technical University of Cluj-Napoca, Cluj-Napoca, Romania 
Let (xn),>0 be the sequence defined by x9=0,x1=1,x2=1 and Xn+3=%Xn+2t%Xn+1 +X +n, Vn > 0. Prove that the series 


a Xn Ta (6 
y 50 converges and find its sum. 


n=1 
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